Connect with us

Technology

The Movie Gen Meta model provides realistic video with sound, so we can finally have infinite Moo Deng

Published

on

No one really knows yet what generative video models are good for, but that does not stop firms like Runway, OpenAI, and Meta from investing thousands and thousands of their development. The latest version of Meta is titled Movie Genand true to its name, it turns text prompts into relatively realistic video with sound… but luckily no voice yet. And it’s sensible that they do not post it publicly.

Movie Gen is definitely a set (or “cast” as they call it) of basic models, the most important of which is the text-to-video bit. Meta claims it outperforms the likes of Runway Gen3, the newest LumaLabs release, and Kling 1.5, although as all the time, this type of thing shows more that they are playing the identical game than Movie Gen winning. The specs can be present in Meta’s release article describing all components.

Sound is generated to match the content of the video, adding, for instance, engine sounds corresponding to the movements of the automotive, the sound of a waterfall within the background, or thunder mid-video when required. He’ll even add music if he thinks it is vital.

It was trained on “a combination of licensed and publicly available datasets” that they called “proprietary/commercially sensitive” and provided no further details about it. We can only guess, which means there are various videos on Instagram and Facebook, in addition to some partner materials and rather more, that usually are not properly shielded from scrapers – i.e. “publicly available”.

However, Meta is clearly aiming here not only to say the “state-of-the-art” crown for a month or two, but for a practical, soup-to-nuts approach during which a quite simple material can be changed into a solid end product, a natural language prompt. Things like “imagine me as a baker baking a shiny hippopotamus-shaped cake during a storm.”

For example, one in every of the sticking points with these video generators is how difficult they have an inclination to be to edit. If you request a video of an individual crossing the road and also you realize that you just want the person to walk from right to left, not left to right. There’s a very good probability the entire shot will look different when you repeat the prompt with additional instruction. The meta adds an easy, text-based editing method where you can just say “change the background to a busy intersection” or “change her clothes to a red dress” and she is going to attempt to make that change, nevertheless it’s a change.

Image credits:Meta

Camera movements are also generally understood, and things like “tracking shot” and “panning left” are taken into consideration when generating video. It’s still quite clunky in comparison with actual camera controls, nevertheless it’s significantly better than nothing.

The model limitations are a bit strange. It generates video with a width of 768 pixels, a dimension familiar to most from the famous but outdated 1024×768 resolution, but which can be thrice the width of 256, so it plays well with other HD formats. The Movie Gen system upscales this resolution to 1080p, which is the source of the claim that it produces this resolution. Not entirely true, but we’ll leave them alone because upscaling is surprisingly effective.

Oddly enough, it generates as much as 16 seconds of video… at 16 frames per second, a frame rate that nobody in history has ever wanted or asked for. However, you can also record 10 seconds of video at 24 frames per second. Lead with it!

As for why it doesn’t play the voice… well, there are probably two reasons. First of all, it is extremely difficult. Generating speech is now easy, but matching it to lip movements and lip movements to faces is a rather more complicated proposition. I do not blame them for leaving it until later because it could have been a one-minute fail. Someone might say, “generate a clown delivering the Gettysburg address by riding around on a little bicycle” – nightmare fuel primed for popularity.

The second reason might be political: releasing a deepfake generator a month before the major elections is… not one of the best for optics. A practical preventive step is to barely limit its capabilities so that if malicious actors try to make use of it, it requires real work on their part. You can actually mix this generative model with a speech generator and an open mouth sync generator, but you can’t just generate a candidate making crazy claims.

“Movie Gen is currently purely an AI research concept, and even at this early stage, security is a top priority, as it is with all of our generative AI technologies,” a Meta representative said in response to TechCrunch’s questions.

Unlike, say, Llama’s large language models, Movie Gen is not going to be publicly available. You can replicate these techniques to some extent by following the research paper, however the code is not going to be published apart from the “baseline evaluation prompt dataset,” a record of the prompts used to generate the test videos.

This article was originally published on : techcrunch.com
Continue Reading
Advertisement
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Technology

The new investment by Indian HealthKart is valued at $500 million

Published

on

By

Indian omnichannel nutrition startup HealthKart raised $153 million this 12 months in considered one of the country’s largest consumer startup deals, valuing the corporate at about $500 million, based on a source aware of the matter.

The new investment was co-led by private equity firms ChrysCapital and Motilal Oswal, with Avendus Capital serving as financial advisor. A91 Partners and Asset Manager Neo Group also participated within the new investment.

According to people aware of the matter, some existing investors sold their shares to new sponsors. HealthKart counts Peak XV, Temasek, Sofina and wealth manager IIFL amongst its supporters.

Gurugram-based HealthKart reported revenue of $118.5 million for the 12 months ending March 2024, strengthening its position as India’s largest consumer nutrition platform. The startup sells protein supplements and health accessories.

The 13-year-old startup, which grew out of online pharmacy startup 1MG, said it was also buying back $6.5 million value of stock from employees. The startup generated profitable EBITDA within the financial 12 months ending in March.

“The Indian sports nutrition market, currently underpenetrated, is expected to expand due to increased fitness awareness and the growing importance of nutrition and protein,” Arpit Vinayak, vice chairman of ChrysCapital, said in a press release.

This article was originally published on : techcrunch.com
Continue Reading

Technology

World of Warcraft turns 20 years old

Published

on

By

Blizzard Entertainment first released World of Warcraft in November 2004, so The New York Times celebrated an anniversary describing how 20 years later we will still see the influence of online multiplayer role-playing games.

First, while multiplayer games and early social networks like MySpace already existed, WoW provided an actual preview of a future where anyone could connect with friends and strangers on the (*20*). Second, the sport made billions of dollars on a business model that combined monthly subscriptions with in-game purchases (including pets and animals that players could ride), becoming a large money cow for Blizzard and paving the way in which for future online business models.

The game also appeared immortal memesattracted celebrity fans and suggested epidemiologists argue that an incident involving the uncontrolled spread of a fantasy disease could possibly be investigated to realize insight into real-world epidemics.

Other than that, I didn’t think the movie was that bad.

This article was originally published on : techcrunch.com
Continue Reading

Technology

Exploration Company is raising $160 million to create Europe’s answer to SpaceX Dragon

Published

on

By

Nyx orbital vehicle The Exploration Company

Only two firms currently deliver cargo to and from the International Space Station, and each are based within the United States. Exploration Companywhich operates in Germany, France and Italy, wants to change that: it has just closed a big round of financing to proceed its mission to construct Europe’s first reusable space capsule.

The $160 million Series B round will fund further development of the Nyx spacecraft, which is able to find a way to carry 3,000 kilograms of cargo to Earth and back. The company, founded three years ago by aerospace engineers Hélène Huby, Sebastien Reichstat and Pierre Vine, goals to conduct Nyx’s maiden flight to and from the ISS in 2028.

“We are the first company in the world where, for the first time, it is funded primarily by private investors,” Huby said in a recent interview. This contrasts with SpaceX’s Dragon capsule, which it said was “primarily funded by NASA.”

With the brand new financing, led by Balderton Capital and Plural, the startup’s total funding now stands at over $208 million. Bessemer Venture Partners, NGP Capital and two European sovereign funds, French Tech Souveraineté and DeepTech & Climate Fonds, also participated within the Series B.

“We have managed to deliver on the promises we have made over the last three years,” Huby said. “We were able to hit our cash target every quarter… Investors could see that we were basically able to deliver on time, on cost and with quality.”

The startup has partnered with the European Space Agency (ESA), which has recognized the necessity to support indigenous space launch and transportation capabilities. Earlier this 12 months, Exploration Company was awarded a research contract value roughly €25 million ($27 million) to develop cargo return services. This contract will run until 2026, after which additional competitive contracts are expected to follow. ESA’s goal is to launch no less than one capsule to the ISS in 2028.

The structure of the contract, called the LEO Cargo Return Service Contract, is similar to the NASA Commercial Orbital Return Transportation Services program, which the agency launched in 2006. This program resulted in multi-billion-dollar transportation contracts with SpaceX and Orbital Sciences Corporation (now Northrop Grumman).

It’s a promising start, but equally promising is the potential The Exploration Company sees on the industrial side. About 90% of the startup’s $770 million order book comes from private station developers Vast, Axiom Space and Starlab, according to the most recent reports.

The first Exploration Company demonstration vehicle was launched this summer on the maiden flight of Ariane 6, but it surely was not deployed due to an issue with the rocket’s upper stage. A second, smaller-scale demonstration mission, called Mission Could, is scheduled to launch aboard a SpaceX Falcon 9 next 12 months.

“I really respect what SpaceX has accomplished,” Huby said. “We try to get the most out of it, we are inspired by what they achieved. However, we also believe that the world needs more competition and we want to build an alternative step by step. We are very aware that we are late, that we are much smaller, etc., but we have to start.”

This article was originally published on : techcrunch.com
Continue Reading
Advertisement

OUR NEWSLETTER

Subscribe Us To Receive Our Latest News Directly In Your Inbox!

We don’t spam! Read our privacy policy for more info.

Trending