Technology
Many AI model safety assessments have significant limitations
Despite the growing demand for AI security and accountability, today’s tests and benchmarks will not be enough, a brand new report finds.
Generative AI models—models that may analyze and generate text, images, music, video, and more—are coming under increasing scrutiny for his or her tendency to make mistakes and usually behave unpredictably. Now, organizations from public sector agencies to big tech firms are proposing recent benchmarks to check the safety of those models.
At the tip of last yr, the startup Scale AI created lab dedicated to assessing how well models adhere to security guidelines. This month, NIST and the U.K. AI Safety Institute released tools designed to evaluate model risk.
However, these tests and model testing methods could also be insufficient.
The Ada Lovelace Institute (ALI), a British non-profit organization dedicated to artificial intelligence research, conducted test who interviewed experts from academic, civil society, and vendor modeling labs and examined recent research on AI security assessments. The co-authors found that while current assessments will be useful, they should not comprehensive, will be easily fooled, and don’t necessarily provide guidance on how models will perform in real-world scenarios.
“Whether it’s a smartphone, a prescription drug, or a car, we expect the products we use to be safe and reliable; in these sectors, products are rigorously tested to ensure they’re safe before being deployed,” Elliot Jones, a senior researcher at ALI and co-author of the report, told TechCrunch. “Our research aimed to examine the limitations of current approaches to assessing AI safety, assess how assessments are currently being used, and explore their use as a tool for policymakers and regulators.”
Benchmarks and red teaming
The study’s co-authors first surveyed the tutorial literature to determine an summary of the harms and risks that current models pose and the state of existing assessments of AI models. They then interviewed 16 experts, including 4 employees of unnamed technology firms developing generative AI systems.
The study revealed that there’s wide disagreement across the AI industry on the perfect set of methods and taxonomies for evaluating models.
Some evaluations only tested how well the models matched benchmarks within the lab, not how the models might impact real-world users. Others were based on tests designed for research purposes, not on evaluating production models—yet vendors insisted on using them in production.
We’ve written before concerning the problems with AI benchmarking. This study highlights all of those issues and more.
Experts cited within the study noted that it’s hard to extrapolate a model’s performance from benchmark results, and it’s unclear whether benchmarks may even show that a model has a certain capability. For example, while a model may perform well on a state exam, that doesn’t mean it can have the ability to resolve more open legal challenges.
Experts also pointed to the issue of knowledge contamination, where benchmark results can overstate a model’s performance if it was trained on the identical data it’s being tested on. Benchmarks, in lots of cases, are chosen by organizations not because they’re the perfect assessment tools, but due to their convenience and ease of use, experts said.
“Benchmarks run the risk of being manipulated by developers who may train models on the same dataset that will be used to evaluate the model, which is equivalent to looking at an exam paper before an exam or strategically choosing which assessments to use,” Mahi Hardalupas, a researcher at ALI and co-author of the study, told TechCrunch. “Which version of the model is being evaluated also matters. Small changes can cause unpredictable changes in behavior and can override built-in safety features.”
The ALI study also found problems with “red-teaming,” the practice of getting individuals or groups “attack” a model to discover gaps and flaws. Many firms use red-teaming to judge models, including AI startups OpenAI and Anthropic, but there are few agreed-upon standards for red-teaming, making it difficult to evaluate the effectiveness of a given effort.
Experts told the study’s co-authors that finding individuals with the correct skills and experience to steer red teaming efforts will be difficult, and the manual nature of the method makes it expensive and labor-intensive, a barrier for smaller organizations that don’t have the mandatory resources.
Possible solutions
The foremost the reason why AI rankings have not improved are the pressure to release models faster and the reluctance to run tests that might cause issues before launch.
“The person we spoke to who works for a foundation modeling company felt that there is more pressure within companies to release models quickly, which makes it harder to push back and take assessments seriously,” Jones said. “The major AI labs are releasing models at a speed that outpaces their ability or society’s ability to ensure they are safe and reliable.”
One ALI survey respondent called evaluating models for safety an “intractable” problem. So what hopes does the industry—and those that regulate it—have for solutions?
Mahi Hardalupas, a researcher at ALI, believes there’s a way forward, but it can require greater commitment from public sector entities.
“Regulators and policymakers need to be clear about what they expect from ratings,” he said. “At the same time, the ratings community needs to be transparent about the current limitations and potential of ratings.”
Hardalupas suggests that governments mandate greater public participation in the event of assessments and implement measures to support an “ecosystem” of third-party testing, including programs to offer regular access to any required models and datasets.
Jones believes it could be mandatory to develop “context-aware” assessments that transcend simply testing a model’s response to a command, and as an alternative consider the sorts of users a model might affect (akin to people of a certain background, gender, or ethnicity), in addition to the ways wherein attacks on models could bypass security measures.
“This will require investment in fundamental evaluation science to develop more robust and repeatable evaluations based on an understanding of how the AI model works,” she added.
However, there’s never a guarantee that a model is protected.
“As others have noted, ‘safety’ is not a property of models,” Hardalupas said. “Determining whether a model is ‘safe’ requires understanding the contexts in which it is used, to whom it is sold or shared, and whether the safeguards that are implemented are appropriate and robust to mitigate those risks. Baseline model assessments can serve exploratory purposes to identify potential risks, but they cannot guarantee that the model is safe, much less ‘completely safe.’ Many of our interviewees agreed that assessments cannot prove that a model is safe and can only indicate that the model is unsafe.”
Technology
OpenAI may change its nonprofit structure next year
It looks increasingly likely that OpenAI will soon change its complex corporate structure.
Reports earlier this week suggested the AI company was in talks to boost $6.5 billion at a pre-funding valuation of $150 billion. Now Reuters reports that The deal is contingent on OpenAI successfully restructuring and lifting the profit cap for investors.
In fact, based on FortuneCo-founder and CEO Sam Altman told employees at a company-wide meeting that OpenAI’s structure will likely change next year, bringing it closer to a standard for-profit business. OpenAI is currently structured in order that its for-profit arm is controlled by a nonprofit, which seems to have frustrated investors.
“We remain focused on building AI that benefits everyone, and as we’ve said before, we’re working with our board to ensure we’re best positioned to deliver on our mission,” OpenAI said in an announcement. “The nonprofit is core to our mission and will continue to exist.”
Technology
LinkedIn games are really cool
I actually have a weakness that I’m ashamed of, and it isn’t that I’ve watched all of Glee (yes, even the terrible later seasons) or that I’ve read an incredible amount of Harry Potter fan fiction in my life.
My little weakness is playing LinkedIn games.
To answer the plain query: Wait, LinkedIn has games? Yes. In May, LinkedIn launched three puzzle games via LinkedIn News, like New York Times game knockoffs. There’s the logic puzzle Queens (my favorite), the word game Crossclimb (pretty good), and the association game Pinpoint (not great, but oh well).
LinkedIn is taking the classic tech strategy of seeing what works for one more company after which trying to copy that success, even when it could appear odd to play games on knowledgeable networking platform. But it’s no wonder NYT Games inspired that inspiration. In a way The New York Times is a gaming company now – from December 2023 users I spent more time within the NYT Games app than within the news app.
LinkedIn isn’t alone. Everyone has games now. Apple News. Netflix. YouTube. There are so many games we are able to take pleasure in. And yet, once I finish my various New York Times puzzles, I still want more. It’s not that I feel like playing Crossclimb LinkedIn before Connections, however the games are adequate to provide me that sweet dopamine rush.
I often play LinkedIn games in the course of the workday (sorry to my boss). Sometimes it’s because I’m on LinkedIn to envision facts or look up a source, but then I remember I can spare a number of minutes for slightly game. Other times, my mind is foggy from gazing the identical draft of an article for too long, and taking a break to resolve a colourful Queens puzzle makes it easier to return and revisit that Google doc.
But it turns on the market’s a scientific explanation for why we love these quick, once-a-day puzzles a lot.
I recently spoke with DeepWell DTx cofounder Ryan Douglas, whose company relies on the concept playing video games (moderately) can have a positive impact on mental health. In some cases, the transient distraction of a game can pull us out of a negative thought spiral or help us approach an issue from a brand new perspective.
“If you’re playing Tetris, for example, you can’t have a long conversation in your head about how terrible you are, how much you suck, what’s going to happen next week, and so on,” Douglas told TechCrunch.
On a neurobiological level, Douglas explained that after we play, we activate the limbic system within the brain, which is answerable for coping with stress. But even when these stressors are simulated, they accustom the brain to coping with that stress in some ways.
“You start learning on a subconscious level, creating new neural pathways at an accelerated rate and preferentially selecting them on a subconscious level to deal with those problems in the future,” he said. “If you deal with (the stressor) in that particular environment, you gain agency. You have control.”
That’s to not say we must always play Pokémon all day—the video game development tools DeepWell creates are approved for therapeutic use in 15-minute doses. Maybe that’s why we’re so infatuated with games like Wordle, and other games The New York Times (and LinkedIn) has written which have a finite ending. You solve one puzzle a day, and then you definately move on to the following.
Wordle creator Josh Wardle spoke to TechCrunch about his viral success even before The New York Times picked up his game.
“I’m a little suspicious of apps and games that want your endless attention — I worked in Silicon Valley, for example. I know why they do that,” Wardle said. “I think people have an appetite for things that clearly don’t want anything from you.”
But Wardle is correct—after all my beloved LinkedIn games want something from me: my attention. And to be honest, I’ve spent rather a lot more time on LinkedIn in recent months than I ever have.
According to LinkedIn’s data, my behavior isn’t an anomaly. The company found that latest player engagement has increased by about 20% week over week because the starting of July. LinkedIn has also seen strong traction in users starting conversations after playing games. After you finish a game, you may see which of your connections also played, which I imagine some people see as a chance to #network. I don’t do this, but on the other hand, most of my LinkedIn conversations are just me messaging my friends “hi” because for some reason I find that funny.
So go on LinkedIn and have a good time as much as you may… after which, about 4 minutes later, return to the relentless grind of worldwide capitalism.
Technology
These two friends created a simple tool to transfer playlists between Apple Music and Spotify, and it works great
Last yr, I had the misfortune of losing all my playlists after I moved from Apple Music to Spotify. For me, playlists are necessary. They’re snapshots of a certain period in your life; possibly your summer of 2016 had a soundtrack. But traditionally, streaming music services don’t make it easy to take your playlists with you to other platforms.
You can imagine how joyful I used to be to see that Apple Music has created latest playlist uploader through the Data Transfer Initiative (DTI), a group founded by Apple, Google, and Meta to create data transfer tools. The Digital Markets in Europe Act requires these designated “gatekeepers” to fund data transfer tools as a part of a broader solution to Big Tech’s strategy of blocking users from their platforms.
Finally! There was only one big problem. The tools don’t work with the world’s hottest music service, Spotify, which apparently didn’t catch the wave of knowledge transfer (or possibly the regulator doesn’t tell them to). The DTI tool only transfers data between Apple Music and YouTube Music, making it much less useful for most individuals.
DTI Executive Director Chris Riley can be fed up with Big Tech’s blocking policies. He’s trying to get more firms to join the negotiations and make their services more portable.
“Over the last decade, we’ve kind of blended into this world, just feeling trapped,” Riley told TechCrunch. “I don’t think enough people know that this is something they need to know.”
With DTI limitations in mind, Riley suggested I move my playlists from Apple Music to Spotify using Soundfree third-party tool. Instead of working directly with streaming services, Soundiiz builds portability tools through existing APIs and acts as a translator between services. Within minutes, I used to be able to connect my accounts, transfer my playlists, and start listening to my old Apple Music playlists on Spotify. It was amazing and easy.
Soundiiz allows you to transfer playlists between Apple Music, Spotify, YouTube Music, Amazon Music, Tidal, Deezer, SoundCloud, and 20 other streaming services I’ve never heard of. There’s a simple user interface for connecting streaming services and choosing the playlists you would like to transfer, including ones another person has created.
The story behind Soundiiz may explain why it works so well and cheaply. It was created in 2013 by two friends from France, Thomas Magnano and Benoit Herbreteau, who loved listening to music while coding together. In the evenings, they decided to create a music search interface with input from everywhere in the web. In the method, they created a useful tool.
They never created a music search interface, however the playlist uploader became Soundiiz.
“I had to manipulate the API and test the fit between services. And while I was doing that, I was creating playlists and moving them between services, just for me internally,” Magnano told TechCrunch. “I presented this feature to a colleague of mine and we thought, ‘Oh, this is useful to me; maybe it’s useful to someone else.’”
In 2015, Soundiiz got its big break when it partnered with Tidal, the music service founded by Jay-Z. The music platform was trying to make it easier for people to leave Spotify and join Tidal with all the identical playlists, and Soundiiz helped with that. But Magnano says they made sure Tidal also let people export playlists, not only import them — something they require from every music service API they work with.
Then a lot more people began using the service, and the founders made Soundiiz their full-time job, but they kept their values. The two founders make a living from Soundiiz, but they tell TechCrunch they’re “not looking to get rich.” Magnano says Soundiiz has never sought outside investment to keep prices low, and the founders retain control over their project.
There are limitations to the free Soundiiz though – a number of the longer playlists might be shortened (limited to 200 songs). You even have to transfer playlists one after the other, and every one takes about a minute, so transferring a dozen or so playlists can take a while. Soundiiz offers a premium plan ($4.50 monthly, which you’ll cancel after transferring) to get around these limitations.
The two founders are still the one employees of Soundiiz, regardless that the corporate has grown: Soundiiz has helped hundreds of thousands of individuals move over 220 million playlists over the past 10 years. According to Magnano, they’ve never spent a dime on marketing, but he says they’ve never had to.
“If you were to Google ‘how to transfer Deezer to Spotify’ in 2012, there was no answer,” Magnano said. “So Soundiiz became the first result in Google search when we launched, and we’ve been doing great in SEO ever since.”
Magnano says Spotify likely has more to lose than to gain by creating a playlist uploader like Apple and Google, and he doesn’t expect that to change anytime soon. However, he says that every one of those streaming services are aware of what Soundiiz is doing and are okay with it — some even promote it of their FAQs. That said, it’s unlikely that any of them would promote playlist uploaders like Soundiiz greater than this.
-
Press Release6 months ago
CEO of 360WiSE Launches Mentorship Program in Overtown Miami FL
-
Business and Finance3 months ago
The Importance of Owning Your Distribution Media Platform
-
Press Release5 months ago
U.S.-Africa Chamber of Commerce Appoints Robert Alexander of 360WiseMedia as Board Director
-
Business and Finance6 months ago
360Wise Media and McDonald’s NY Tri-State Owner Operators Celebrate Success of “Faces of Black History” Campaign with Over 2 Million Event Visits
-
Film5 months ago
Time Selects Taraji P. Henson to Host ‘Time100 Special’ in 2024 on ABC
-
Press Release6 months ago
Eggstravaganza, Returning to Miramar Regional Park
-
Ben Crump6 months ago
Attorney Ben Crump vs Google Black Minority Lawsuit
-
Fitness5 months ago
At Smartwater Wellness, check in with fitness trainer Shy Lovell