As more publishers end their content licensing agreements with ChatGPT developer OpenAI, a test posted this week by Towing Center for Digital Journalism — how an AI chatbot creates citations (i.e. sources) for publishers’ content — makes for interesting or, well, disturbing reading.

In short, the findings suggest that publishers remain on the mercy of a generative AI tool’s tendency to invent information or otherwise misrepresent it, whether or not they allow OpenAI to index their content or not.

A study from the Columbia Journalism School examined citations produced by ChatGPT after asking it to discover the source of sample citations from various publishers – a few of which had contracts with OpenAI and others who didn’t.

“We selected citations that, when pasted into Google or Bing, resulted in the source article being in the top three results and assessed whether the new OpenAI search engine would correctly identify the article that was the source of each citation,” Tow researchers Klaudia Jaźwińska and Aisvarya Chandrasekar wrote in blog post explaining your approach and summarizing your findings.

“What we found did not make news publishers feel optimistic,” they proceed. “While OpenAI emphasizes its ability to provide users with “timely responses that include links to appropriate online sources,” the corporate makes no express commitment to make sure the accuracy of those citations. This is a noticeable omission for publishers who expect their content to be referenced and accurately represented.”

“Our tests showed that no publisher – regardless of their level of affiliation with OpenAI – was spared from inaccurate representation of their content in ChatGPT,” they added.

Unreliable source

Researchers say they found “numerous” cases where ChatGPT misquoted publishers’ content, also finding what they call “response accuracy ghosting.” So, while they found “a few” completely correct quotes (i.e. ChatGPT accurately returned the publisher, date, and URL of the shared quote), there have been “many” quotes that were completely unsuitable; and “some” that fall somewhere in between.

In short, ChatGPT quotes appear to be an unreliable mix. The researchers also found only a few cases where the chatbot didn’t show complete confidence in its (erroneous) answers.

Some quotes come from publishers who actively blocked OpenAI search bots. Scientists say they expected problems with creating correct citations in such cases. However, they found that this scenario created one other problem – the bot “rarely” “admitted that it was unable to provide an answer.” Instead, he returned to confabulation to generate some sources (albeit incorrect sources).

“In total, ChatGPT returned partially or completely incorrect responses in 153 cases, although it only confirmed the inability to accurately answer the query seven times,” the researchers said. “Only for these seven results did the chatbot use qualifying words and phrases resembling ‘it seems’, ‘it’s possible’ or ‘perhaps’ or statements resembling ‘I could not locate the precise article.’

They compare this unlucky situation to a regular Internet search, where serps like Google or Bing typically either locate the precise quote and point the user to the web site where they found it, or state that they found no results with an actual match.

“ChatGPT’s lack of transparency around trust in responses may make it difficult for users to assess the validity of a claim and understand which parts of the response they can and cannot trust,” they argue.

They suggest that publishers may additionally face reputational risk from incorrect citations, in addition to business risk from sending readers elsewhere.

Decontextualized data

The study also highlights one other issue. This suggests that ChatGPT may essentially reward plagiarism. Researchers cite a case through which ChatGPT misquoted an internet site that plagiarized “deeply reported” New York Times journalism, i.e., by copying and pasting text without attribution because the source of a NYT article – speculating that in such a case, the bot could have generated this false answer to fill the data gap resulting from the lack to look the NYT website.

“This raises serious questions about OpenAI’s ability to filter and check the quality and authenticity of data sources, especially for unlicensed or plagiarized content,” they suggest.

An additional finding that could be of concern to publishers who’ve signed deals with OpenAI is that of their case ChatGPT quotes weren’t at all times reliable either – so letting in bots doesn’t seem to ensure accuracy either.

The researchers argue that the elemental issue is that OpenAI’s technology treats journalism “as context-free content,” apparently whatever the circumstances of its creation.

Another issue noted within the study is the variation in ChatGPT responses. The researchers tested asking the bot the identical query multiple times and located that it “usually returned a different answer each time.” While that is typical of GenAI tools, generally, within the context of citations, such inconsistency is clearly suboptimal in case you care about accuracy.

While Tow’s study was conducted on a small scale – the researchers acknowledge that “more rigorous” testing is required – it’s nevertheless noteworthy given the high-level deals that major publishers are busy making with OpenAI.

If media corporations had hoped that these arrangements would result in special treatment for their content in comparison with competitors, at the least when it comes to ensuring accurate sourcing, this study suggests that OpenAI has not yet ensured such consistency.

While publishers who do not have licensing agreements also block OpenAI bots entirely – perhaps in hopes of at the least increasing traffic when ChatGPT returns content related to their articles – the study can be bleak since the citations might not be accurate of their content. cases too.

In other words, there is no such thing as a guarantee of “visibility” for publishers on OpenAI’s search engine, even in the event that they allow their crawlers to achieve this.

Blocking robots completely doesn’t mean that publishers can protect themselves from reputational risk by avoiding any mention of their articles on ChatGPT. The study found that the bot continued to incorrectly attribute articles to the New York Times, for example, despite the continuing lawsuit.

“Small significant agency”

The researchers concluded that, because it stands, publishers have “little meaningful influence” over what happens to and with their content once ChatGPT gets its hands on it (either directly or, well, not directly).

OpenAI’s response to the research results appeared in a blog post, accusing researchers of conducting an “unusual test of our product.”

“We support publishers and creators by helping ChatGPT’s 250 million weekly users discover high-quality content through summaries, citations, clear links and attributions,” OpenAI also told them, adding: “We have worked with partners to improve the accuracy of in-line citations and respect publisher preferences , including enabling a way for them to appear in search by managing OAI-SearchBot in their robots.txt file. We will continue to improve search results.”

As large enterprises struggle to include AI into their platforms and processes, they’ve run into a problem: Generative AI should have memory and its training data should be consistently updated to be of any practical use. This area is currently called “Live AI” and many startups operate in it, including: Cohere AND Writer. Other, Pathjust raised a $10 million seed round to construct working artificial intelligence systems that the company says think and learn in real time like humans.

The round was led by TQ Ventures, with participation from Kadmos, Innovo, Market One Capital, Id4 and business angels. Another investor in Pathway is Łukasz Kaiser, co-author Transformers and key researcher behind GPT o1 from OpenAI.

Pathway’s offering includes so-called “infrastructure components” that power running AI systems and feed structured and unstructured data, meaning enterprise AI platforms could make decisions based on up-to-date knowledge. Current clients include NATO and the French post office La Poste.

Zuzanna Stamirowska, co-founder and CEO of Pathway, told TechCrunch on a call: “The way deep learning assistants and LLM work is that you take training data and then train models. But the question is: how to deal with knowledge, how to deal with memory? At this point, the LLM is behaving a bit like a very smart intern on his first day of work who has been offered a book to read. But they can’t actually remember it. Besides, it’s not live, it’s static.”

To address this, she said Pathway “enables developers to build a pipeline through which they can feed live data into AI systems.” Right now, we’re doing this in the prompting stage whenever you’re constructing LLM applications or Gen AI applications.

Stamirowska — who’s moving to Menlo Park, California — has assembled a formidable, highly technical team to attain the startup’s goals. Its co-founders are CSO Adrian Kosowski and CTO Jan Chorowski, who previously worked with recent Nobel Prize winner in physics and “godfather of artificial intelligence” Geoff Hinton. Stamirowska herself is the writer of a cutting-edge forecasting model for the complex network occurring in maritime trade, published by the US Academy of Sciences.

“The company started with an idea that popped into my head one sunny morning in Chicago,” she said. “I used to be there accompanying a friend to a research conference on theoretical computer science… We had an argument and I said I had to begin with myself. So I took out my laptop and began writing to people in my network about move things forward. I still remember the taste of coffee from that moment.

I asked her where she sees Pathway in comparison with other startups in the space? “For applications in GenAI engineering and knowledge management, Cohere and Writer appear alongside us in Gartner’s newest quadrants,” she said. “Whereas in enterprise deals we often see Palantir bidding on AI transformation, although they are less product-oriented than we are.”

Commenting in a statement, Schuster Tanger, co-managing partner and co-founder of TQ Ventures, said: “Zuzanna and the Pathway team have cutting-edge knowledge and expertise in one of the most exciting areas of business today… The recent, and almost at least, response from the developer community has been powerful.”

Zomato has raised $1 billion in a so-called qualified institutional placement, completing its first major fundraising since its 2021 IPO.

Friday’s stock exchange filing showed the $30 billion food delivery and fast-trading giant, which leads in each categories, issued about 336.5 million shares at 252.62 kilos ($3) each.

The placement, which began on November 25 and ended on November 28, enjoyed large participation from leading domestic investment funds. The largest investor turned out to be Motilal Oswal, whose family of funds covered 20.81% of the difficulty. ICICI Prudential Funds secured 12.78%, while HDFC and Kotak Funds secured 8.68% and 5.95% respectively.

The timing of the capital raise can be strategic, coming just weeks after rival Swiggy’s $1.35 billion initial public offering earlier this month. Zepto, one other leading fast trading startup, earlier this month secured $350 million in a deal brokered by Motilal Oswal.

Swiggy shares fell 4.1% on Friday, limiting the general gain this week to 12.8%. Zomato shares, offered to investors participating in the QIP at a 5% discount, fell about 1% on Friday but are up 127.7% since the start of the 12 months.

Zomato co-founder and CEO Deepinder Goyal said last month that the corporate, which already has money reserves of $1.3 billion, was searching for additional funds to take care of competitive parity.

The company, which recently reported its second straight quarterly profit, is a frontrunner in India’s fast trading market via Blinkit, competing with well-funded rivals corresponding to Swiggy, Zepto and BigBasket in a sector expected to generate greater than $6.5 billion in annual revenue .

“We see that the fast trading industry will go through a phase of increased competition over the next 6-12 months. Companies operating on the fast trading market are looking for capital or have already obtained it. Four new names including Flipkart, Reliance, BigBasket and Amazon are looking to enter the fast trading space,” Bank of America analysts said in a note.

“The first mover advantage is what matters in this space, and given that the TAM is around 30 million households (330 million households in India), we believe it is logical that market leader Zomato would want to maintain its leading position c. 40% share.”