Google DeepMind’s Genie 


  • Recently Google DeepMind has introduced Genie which is a new model that can generate interactive video games from just a text or image prompt.
  • Genie can generate games without any prior training on game mechanics (which are essentially rules, elements, and processes that make up a game).

What is Genie?

  • According to the official Google DeepMind blog post, Genie is a foundation world model which is trained on videos sourced from the Internet.
  • The model can generate an endless variety of playable (action-controllable) worlds from synthetic images, photographs, and also even sketches.
  • The research paper ‘Genie: Generative Interactive Environments’ states that Genie is the first generative interactive environment which has been trained in an unsupervised manner from unlabelled internet videos.
  • When it comes to size, Genie stands at nearly 11B parameters and consists of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model.
  • These technical specifications let the Genie act in generated environments on a frame-by-frame basis even in the absence of training, labels, or any other domain-specific requirements.

What does Genie do?

  • According to the research paper published, Genie is a new kind of generative AI that enables anyone even children to dream up and step into generated worlds similar to human-designed simulated environments.
  • Genie can be prompted to generate a various set of interactive and controllable environments although it is trained on video-only data.
  • In simple words, we have seen numerous generative AI models which can produce creative content with language, images and even videos.
  • It is a breakthrough as it makes playable environments from a single image prompt.
  • According to Google DeepMind, Genie can be prompted with images that it has never seen.
  • These images include real world photographs, sketches, allowing people to interact with their imagined virtual worlds.
  • This is also known as a foundation world model.
  • When it comes to training, the research paper highlights that they focus more on videos of 2D platformer games and also robotics.
  • Genie is trained on a general method which allows it to function on any type of domain, and it is scalable to even larger Internet datasets.

Why is it important?

  • The standout aspect of Genie is that its ability to learn and reproduce controls for in-game characters exclusively from internet videos.
  • This is important because internet videos do not have labels about the action that is performed in the video, or even which part of the image should be controlled.
  • Genie also learns not only which parts of an observation are generally controllable, but also infers diverse latent actions that are consistent across the generated environments.
  • According to Google DeepMind, the most distinct aspect of this Genie model is that it allows you to create an entire new interactive environment from a single image.
  • This opens up many possibilities, especially new ways to create and also to step into virtual worlds.
  • To demonstrate this, the researchers have created an in image using text-to-image model Imagen 2 and then used it as a prompt to create virtual worlds.
  • The same can also be done with sketches.
  • With Genie, anyone will now be able to create their own entirely imagined virtual worlds.
  • Apart from that, the model’s ability to learn and develop new world models signals a significant leap towards general AI agents (an independent programme or entity that interacts with its environments by perceiving its surroundings via sensors).

The post Google DeepMind’s Genie appeared first on Vajirao IAS.


Source link

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *