Google is trying to impress with Gemini, its flagship suite of generative AI models, applications and services.

So what are Gemini? How can you use it? And how does it compare to the competition?

To help you sustain with the latest Gemini developments, we have created this handy guide, which we’ll keep updating as new Gemini models, features, and news about Google’s plans for Gemini grow to be available.

What is Gemini?

Gemini is owned by Google long promised, a family of next-generation GenAI models developed by Google’s artificial intelligence labs DeepMind and Google Research. It is available in three flavors:

• Gemini Ultrathe best Gemini model.
• Gemini Pro“lite” Gemini model.
• Gemini Nanoa smaller “distilled” model that works on mobile devices like the Pixel 8 Pro.

All Gemini models were trained to be “natively multimodal” – in other words, able to work with and use greater than just words. They were pre-trained and tuned based on various audio files, images and videos, a big set of codebases and text in various languages.

This distinguishes Gemini from models akin to Google’s LaMDA, which was trained solely on text data. LaMDA cannot understand or generate anything beyond text (e.g. essays, email drafts), but this isn’t the case with Gemini models.

What is the difference between Gemini Apps and Gemini Models?

Google, proving once more that it has no talent for branding, didn’t make it clear from the starting that Gemini was separate and distinct from the Gemini web and mobile app (formerly Bard). Gemini Apps is solely an interface through which you can access certain Gemini models – consider it like Google’s GenAI client.

Incidentally, Gemini applications and models are also completely independent of Imagen 2, Google’s text-to-image model available in a few of the company’s development tools and environments.

What can Gemini do?

Because Gemini models are multimodal, they will theoretically perform a spread of multimodal tasks, from transcribing speech to adding captions to images and videos to creating graphics. Some of those features have already reached the product stage (more on that later), and Google guarantees that each one of them – and more – can be available in the near future.

Of course, it is a bit difficult to take the company’s word for it.

Google seriously fell in need of expectations when it got here to the original Bard launch. Recently, it caused a stir by publishing a video purporting to show the capabilities of Gemini, which turned out to be highly fabricated and kind of aspirational.

Still, assuming Google is kind of honest in its claims, here’s what the various tiers of Gemini will give you the option to do once they reach their full potential:

Gemini Ultra

Google claims that Gemini Ultra – thanks to its multimodality – may help with physics homework, solve step-by-step problems in a worksheet and indicate possible errors in already accomplished answers.

Gemini Ultra can be used for tasks akin to identifying scientific articles relevant to a selected problem, Google says, extracting information from those articles and “updating” a graph from one by generating the formulas needed to recreate the graph with newer data.

Gemini Ultra technically supports image generation as mentioned earlier. However, this feature has not yet been implemented in the finished model – perhaps because the mechanism is more complex than the way applications akin to ChatGPT generate images. Instead of passing hints to a picture generator (akin to DALL-E 3 for ChatGPT), Gemini generates images “natively” with no intermediate step.

Gemini Ultra is accessible as an API through Vertex AI, Google’s fully managed platform for AI developers, and AI Studio, Google’s online tool for application and platform developers. It also supports Gemini apps – but not totally free. Access to Gemini Ultra through what Google calls Gemini Advanced requires a subscription to the Google One AI premium plan, which is priced at $20 monthly.

The AI ​​Premium plan also connects Gemini to your broader Google Workspace account—think emails in Gmail, documents in Docs, presentations in Sheets, and Google Meet recordings. This is useful, for instance, when Gemini is summarizing emails or taking notes during a video call.

Gemini Pro

Google claims that Gemini Pro is an improvement over LaMDA by way of inference, planning and understanding capabilities.

Independent test by Carnegie Mellon and BerriAI researchers found that the initial version of Gemini Pro was actually higher than OpenAI’s GPT-3.5 at handling longer and more complex reasoning chains. However, the study also found that, like all major language models, this version of Gemini Pro particularly struggled with math problems involving several digits, and users found examples of faulty reasoning and obvious errors.

However, Google promised countermeasures – and the first one got here in the type of Gemini 1.5 Pro.

Designed as a drop-in substitute, Gemini 1.5 Pro has been improved in lots of areas compared to its predecessor, perhaps most notably in the amount of information it could actually process. Gemini 1.5 Pro can write ~700,000 words or ~30,000 lines of code – 35 times greater than Gemini 1.0 Pro. Moreover – the model is multimodal – it isn’t limited to text. Gemini 1.5 Pro can analyze up to 11 hours of audio or an hour of video in various languages, albeit at a slow pace (e.g., looking for a scene in an hour-long movie takes 30 seconds to a minute).

Gemini 1.5 Pro entered public preview on Vertex AI in April.

An additional endpoint, Gemini Pro Vision, can process text images – including photos and videos – and display text according to the GPT-4 model with Vision OpenAI.

Using Gemini Pro with Vertex AI. Image credits: Twins

Within Vertex AI, developers can tailor Gemini Pro to specific contexts and use cases through a tuning or “grounding” process. Gemini Pro can be connected to external third-party APIs to perform specific actions.

AI Studio includes workflows for creating structured chat prompts using Gemini Pro. Developers have access to each Gemini Pro and Gemini Pro Vision endpoints and might adjust model temperature to control creative scope and supply examples with tone and elegance instructions, in addition to fine-tune security settings.

However, when Gemini 1.5 Pro leaves the preview in Vertex, the model will cost $0.0025 per character, while the output will cost $0.00005 per character. Vertex customers pay per 1,000 characters (roughly 140 to 250 words) and, for models like the Gemini Pro Vision, per image ($0.0025).

Let’s assume a 500-word article incorporates 2,000 characters. To summarize this text with the Gemini 1.5 Pro will cost $5. Meanwhile, generating an article of comparable length will cost $0.1.

Pricing for the Ultra has not yet been announced.

The National Institute of Standards and Technology (NIST), an agency of the U.S. Department of Commerce that develops and tests technologies for the U.S. government, businesses and most of the people, announced on Monday the launch of NIST GenAI, a brand new program led by NIST to judge generative technologies Artificial intelligence technologies, including artificial intelligence generating text and pictures.

NIST GenAI will publish benchmarks, help create systems for detecting “content authenticity” (i.e., deep-check false information), and encourage the event of software that detects the source of false or misleading information generated by artificial intelligence, NIST explains on newly launched NIST GenAI website and press release.

“The NIST GenAI program will publish a series of challenges designed to assess and measure the capabilities and limitations of generative artificial intelligence technologies,” the press release reads. “These assessments will be used to identify strategies to promote information integrity and guidance for the safe and responsible use of digital content.”

The first NIST GenAI project is a pilot study to construct systems that may reliably distinguish human-generated media from AI-generated media, starting with text. (While many services aim to detect deepfakes, research and our own testing have shown that they’re unreliable, especially in terms of text.) NIST GenAI is inviting teams from academia, industry, and research labs to submit “generators” – AI systems to content generation – i.e. “discriminators”, i.e. systems that attempt to discover content generated by artificial intelligence.

Generators within the study must generate summaries given a subject and set of documents, while discriminators must detect whether a given summary is written by artificial intelligence. To ensure fairness, NIST GenAI will provide data vital to coach generators and discriminators; systems trained on publicly available data is not going to be accepted, including but not limited to open models akin to Meta’s Llama 3.

Registration for the pilot will begin on May 1, and the outcomes might be announced in February 2025.

The launch of NIST GenAI and study specializing in deepfakes comes at a time of exponential growth within the variety of deepfakes.

According to data from Clarity, a deepfake detection company, 900% more deepfakes have been created this yr in comparison with the identical period last yr. This causes concern, which is comprehensible. AND last vote from YouGov discovered it 85% of Americans said they were concerned regarding the spread of misleading deepfakes on the Internet.

The launch of NIST GenAI is an element of NIST’s response to President Joe Biden’s Executive Order on Artificial Intelligence, which sets rules requiring AI firms to be more transparent about how their models perform and establishes quite a lot of recent standards, including for labeling AI-generated content intelligence .

This can be NIST’s first AI-related announcement following the appointment of Paul Christiano, a former OpenAI researcher, to the agency’s AI Security Institute.

Christiano was a controversial alternative as a consequence of his “doomeristic” views; he once predicted that “there is a 50% chance that the development of artificial intelligence will end in (the destruction of humanity).” Criticsreportedly including scientists at NIST, they fear that Cristiano may encourage the AI ​​Security Institute to concentrate on “fantasy scenarios” reasonably than realistic, more immediate threats from artificial intelligence.

NIST says NIST GenAI will report on the work of the AI ​​Security Institute.

In recent years, the specter of job loss because of artificial intelligence has develop into a reality, especially for younger Americans.