The above image was made by 
Recently I wrote a proof-of-concept vector search app using Pyscript. It takes any detailed question or description of any conspiracy-related topic as input and outputs a list of the most relevant news article summaries from the WantToKnow.info news archive. Yesterday I wrote about including more features in the Pandas dataframe the app is built on. Today I'll share the trials and tribulations of AI integration.
There are about 12,970 valid entries in the archive. Right now, the app looks at vectors of every one of these news articles and compares these to a vector computed from a user query to find a list of the most relevant items. I'd like to then feed these news items into gpt-4o to generate a summary brief of the material. To keep things secure, the user would input their OpenAI API key and click a button to generate and display the brief.
The AI functionality may or may not turn out to be useful. But the project gives me a chance to get to know the OpenAI API, which is something I've been meaning to do. Hitting the endpoint turned out to be super easy in a Jupyter notebook. First, I plugged a query into my app:
Forests are magical. Mushrooms are amazing and mysterious. They can also be used to solve problems. Building materials. Soil remediation. Curing treatment resistant depression. Anything is possible with fungi.
This query returned a list of 10 articles like this:
ArticleId: 11489
Title: Netflix documentary 'Fantastic Fungi' explores the many magical properties of mushrooms
PublicationDate: 2021-08-16
Publication: San Francisco Chronicle
Links: https://www.sfgate.com/streaming/article/Netflix-documentary-Fantastic-Fungi-mushrooms-16383646.php
Priority: 900
url: https://www.wanttoknow.info/a-netflix-documentary-fantastic-fungi-explores-many-magical-properties-mushrooms
Summary: Fungi have been around for billions of years, setting the stage for humanity by supporting, carrying and converting life. But for complex political reasons, these organisms are still shrouded in mystery. One man, however, is determined to lift the veil on the magical world of mushrooms. Enter Stamets, a bespectacled author and researcher whose mission to decode nature’s hidden language and explore “altered states of consciousness is chronicled in the documentary “Fantastic Fungi,” which was recently made available to stream on Netflix. While the film aims to destigmatize hallucinogenic mushrooms, it also demonstrates why we should legitimize the studies of all mushrooms. Contemporary experts in neurology, psychiatry and biology in the film show that fungal genomes can solve a host of mental, physical and environmental problems. From healing bacterial infections to cleaning petroleum spills, fungi possess unique, almost godlike properties that are otherwise unseen in nature. For instance, lion’s mane, an edible white mushroom that tastes like lobster, stimulates nerves in order to grow, suggesting that it could potentially cure degenerative diseases like Alzheimer’s. Ultimately, when Stamets discusses altered states of consciousness, it’s ... about accepting a different state of being. For some people especially those who live in pain " the film posits that mushrooms can be the answer they’ve been looking for. "
Note: Explore a treasure trove of concise summaries of incredibly inspiring news articles which will inspire you to make a difference.
ArticleId: 10866
Title: Mushrooms Clean Up Toxic Mess, Including Plastic. So Why Arent They Used More?
PublicationDate: 2019-03-05
Publication: Yes! Magazine
Links: https://www.yesmagazine.org/environment/2019/03/05/mushrooms-clean-up-toxic-mess-including-plastic-why-arent-they-used-more/
Priority: 932
url: https://www.wanttoknow.info/a-mushrooms-clean-up-toxic-mess-including-plastic-so-why-arent-they-used-more
Summary: When wildfires burned across Northern California in October 2017, they killed at least 43 people and displaced another 100,000. The human toll alone was dire, but the fires also left behind a toxic mess. The charred detritus of paint, pesticides, cleaning products, electronics, pressure-treated wood, and propane tanks left a range of pollutants in the soilincluding arsenic, asbestos, copper, hexavalent chromium, lead, and zinc. In Sonoma County, a coalition of fire remediation experts, local businesses, and ecological activists mobilized to cleanse the foundations of burned-out buildings with mushrooms. The Fire Remediation Action Coalition placed more than 40 miles of wattlesstraw-filled, snakelike tubes designed to prevent erosioninoculated with oyster mushrooms around parking lots, along roads, and across hillsides. Their plan? The tubes would provide makeshift channels, diverting runoff from sensitive waterways. The mushrooms would do the rest. The volunteers, led by Sebastopol-based landscape professional Erik Ohlsen, are advocates for mycoremediation, an experimental bioremediation technique that uses mushrooms to clean up hazardous waste, harnessing their natural ability to use enzymes to break down foreign substances. Mushrooms [have been used to] clean up oil spills in the Amazon, boat fuel pollution in Denmark, contaminated soil in New Zealand, and polychlorinated biphenyls, more commonly known as PCBs, in Washington state’s Spokane River. Research suggests mushrooms can convert pesticides and herbicides to more innocuous compounds, remove heavy metals from brownfield sites, and break down plastic. They have even been used to remove and recover heavy metals from contaminated water. Research suggests mushrooms can convert pesticides and herbicides to more innocuous compounds, remove heavy metals from brownfield sites, and break down plastic. "
Note: The stunningly beautiful documentary Fantastic Fungi takes you on an amazing journey through the wild and wonderful world of mushrooms. Explore a treasure trove of concise summaries of incredibly inspiring news articles which will inspire you to make a difference.
These search results were passed to gpt like this:
import openai
from openai import OpenAI
client = OpenAI(api_key="My key")
message = '''My message
'''
payload = "Please generate a 700 word narrative summary of the following information: " + message + '"'
chat_completion = client.chat.completions.create(
messages=[
{
"role": "user",
"content": payload }
],
model="gpt-4o",
)
print(chat_completion)
This returned the following text:
ChatCompletion(id='chatcmpl-9RqdtGBXCaPSUfjcyos9tEeoGNig1', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='The world of mushrooms is enchanting, versatile, and astonishingly powerful. Despite their ancient origins, these organisms hold a plethora of untapped potentials that only now are gaining the appreciation they deserve. This narrative will explore these potentials through various lenses—documentaries, scientific research, ecological initiatives, and newfound medical applications.\n\n"Fantastic Fungi," a documentary available on Netflix, brings the hidden intricacies and marvels of mushrooms into the spotlight. Spearheaded by Paul Stamets, a renowned author and researcher, the film seeks to unravel the complex world of fungi. Stamets' mission is not merely to destigmatize hallucinogenic mushrooms but also to validate the broader study of all fungi. The documentary features insights from contemporary experts in neurology, psychiatry, and biology, who contend that fungal genomes have the potential to solve various mental, physical, and environmental problems.\n\nOne particularly eye-opening segment of "Fantastic Fungi" focuses on the remarkable healing properties of mushrooms. For instance, the lion's mane mushroom, which tastes like lobster, has shown promise in stimulating nerve growth, potentially offering a cure for degenerative diseases like Alzheimer’s. Such discoveries reveal the nearly godlike capabilities of fungi in health, environmental remediation, and much more.\n\nThe transformative abilities of mushrooms extend beyond health benefits. In 2017, Northern California's devastating wildfires left behind a toxic residue, prompting an innovative response. A coalition of fire remediation experts, local businesses, and ecological activists mobilized to use oyster mushrooms to cleanse the pollutant-laden soil. This effort, led by Erik Ohlsen, highlighted the potential of mycoremediation—a technique using fungi to detoxify contaminated environments. These mushrooms have also been employed internationally to clean oil spills, treat boat fuel pollution, and convert pesticides into more harmless compounds.\n\nInterestingly, mycelium, the underlying network of fungi, has been termed "the neurological network of nature" by Stamets. This vast, underground web of fibrous tissue plays a role akin to the human neural network, transmitting nutrients and information across vast distances. In his TED Talk, Stamets elaborates on the fantastic future where fungi could revolutionize fields like medicine, forestry, and pollution control.\n\nThe scientific community is increasingly acknowledging these potentials. Research indicates that mushrooms might even communicate using electrical impulses, forming a language that could consist of up to 50 "words." Professor Andrew Adamatzky's mathematical analysis shows that these electrical patterns bear a striking resemblance to human language, hinting at a complex, collaborative fungal network that might fundamentally reshape our understanding of these organisms.\n\nRecent advancements have also spotlighted the medical potentials of psychedelic mushrooms. The active ingredient in these mushrooms, psilocybin, has shown promising results in treating mental health conditions. COMPASS Pathways, a life sciences company, received a "Breakthrough Therapy" designation from the FDA for its psilocybin treatment targeting treatment-resistant depression. This groundbreaking therapy offers hope to millions affected globally by severe mental illnesses that have proven resistant to conventional treatments.\n\nThe acceptance of psychedelic mushrooms for medical use took a formal leap when Australia became the first country to recognize them as medicine. Starting in July 2023, MDMA and psilocybin gained approval for controlled use, providing a new treatment avenue for conditions like PTSD and treatment-resistant depression. This pivotal move indicates a broader shift toward recognizing the medicinal properties of psychoactive fungi.\n\nWhile mushrooms herald promising medical and environmental breakthroughs, they also face challenges, such as the rise of drug-resistant fungal infections like Candida auris. This pathogen preys on individuals with compromised immune systems, emphasizing the need for vigilant monitoring and innovative treatments.\n\nMushrooms' potential extends even to climate change mitigation through carbon farming practices. Farmers like those at Solidarity Farm in California enhance soil quality and sequester carbon by integrating mushroom-based composting and mulching techniques. These practices have quadrupled carbon storage in their soil, exemplifying mushrooms' role in ecological restoration.\n\nIn essence, mushrooms are transcendent organisms, straddling the boundaries of health, environmental science, and even cognitive communication. As documentaries like "Fantastic Fungi" and groundbreaking research bring their multifaceted roles to light, mushrooms promise to be potent agents of change, capable of addressing some of humanity's most pressing challenges.', role='assistant', function_call=None, tool_calls=None))], created=1716424241, model='gpt-4o-2024-05-13', object='chat.completion', system_fingerprint='fp_729ea513f7', usage=CompletionUsage(completion_tokens=857, prompt_tokens=4521, total_tokens=5378))
As you can see, the AI response is a satisfactory summary. It just needs some cleanup and formatting before being displayed on a page. To move from Jupyter to my Pyscript app, I had hoped to just wrap this functionality in a function called by a button click. Unfortunately, the requests module is unavailable in Pyscript, which means that the openai library that depends on this module is not usable in PyScript. So I spent most of the day trying different approaches to making this new function work. Here's where I'm at now:
async def send(event=None):
Element('summarybrief').write("Send function successfully called.")
userElement = js.document.getElementById('brief')
OPENAI_API_KEY = userElement.value.strip()
oai = "https://api.openai.com/v1/chat/completions"
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {OPENAI_API_KEY}"
}
# Log the API key and headers for debugging
console.log(f"API Key: {OPENAI_API_KEY}")
console.log(f"Headers: {headers}")
# Convert similar_items to string format
global similar_items
similar_items_str = ""
for index, row in similar_items.iterrows():
similar_items_str += f"Title: {row['Title']}\nSummary: {row['Summary']}\nNote: {row['Note']}\n"
payload = f"Please generate a 700 word narrative summary of the following information: '''{similar_items_str}'''"
messages = [
{"role": "user", "content": payload},
]
data = {"model": "gpt-4o", "messages": messages}
# Log the request data for debugging
console.log(f"Data: {data}")
# Use JavaScript fetch
response = await fetch(oai, {
"method": "POST",
"headers": headers,
"body": js.JSON.stringify(data)
})
if not response.ok:
console.log(f"Error: {response.status}")
Element('summarybrief').write(f"Error: {response.status}")
return
response_data = await response.json()
reply = response_data["choices"][0]["message"]["content"]
# Display the response in the summarybrief div
Element('summarybrief').element.innerHTML = reply
# Debugging log
console.log(reply)
For reasons I'm not yet aware of, this code returns a 401 forbidden error. My API key is valid and funded. My headers look perfect. But the error persists, leaving me to suspect that something somewhere is incorrectly formed. But what? And where? At this point, I'm starting to wonder if what I want to do is even possible.
It's interesting to note that the AI query used 857 completion tokens and 4521 prompt tokens. Using gpt-4o costs $5 per million prompt tokens and $15 per million completion tokens. So if I ran the same query about 221 times, it would cost me $5. That's affordable, but it definitely isn't free. Asking for a 500 word summary instead of 700 words might be prudent. I'm still working out the cost/benefit of this new tech.
Read Free Mind Gazette on Substack
Read my novels:
See my NFTs: