Synthetic Media & Generative AI News and Discussions

[Yuli Ban]

That's amazing Yuli! My one thought, though, was, if it is generating that much content every second, it would quickly outrun physical storage, even the best digital storage they have. I wonder what will happen about that? Another interesting thought is, at what point will humanity's entire contribution on the internet be overwhelmed by AI (meaning our contribution to text is in the 0.00001% range)? If this is anything to go by, then it would be less than a year, I'd imagine. The political, social, etc ramifications of this stuff are going to be insane. Even the Trump era will seem tame compared to what will happen I bet. Lol!

Also, I found it fascinating that as this AI has risen, human creative output has dropped. I read in the comments that one individual mentioned they thought it was because people were realizing they are not as good as AI and are dropping out of producing anything. What do you think Yuli (or anyone else that wants to chime in)? That was sort of expected, sort of not for me. I know there are quite a few in the creative community that are only doing creative work as a hobby and/or as a way to make more money on the side, so I could see both those groups dropping out of creative production, for the most part. The only ones that remain will probably be ones that have a huge passion for it, or for those that have a job writing.

1: Hold on there. Sure, GPT-3 generates a novel a second, but most of those words are useless. We're going to need a few more advancements before it really becomes a threat to creativity.

2: On that note, I'm an example of this myself. At first, I considered giving up writing and decided to just work my day-job until synthetic media matured, but I since decided to do the opposite: return to writing and exploit the early days of synthetic media for my own benefit. And then when it was maturing, I'd have loads of material to synthesize

If I can admit anything, knowing just how disruptive AI will be in the next few years and how it'll directly impact my own field has allowed me to adopt a sort of Antiquity-esque mindset not dissimilar to the old Greeks where I've started caring less about the acquisition of wealth even though I know exactly how to break on through (though to be fair, I'm considering going that route again and have been putting it off; the idea has been to exploit AI for that purpose as well). My mind's been stuck on the inevitabilities of synthetic media more than anything else lately.

[Yuli Ban]

[Yuli Ban]

I disagree, but only partially. Current language models aren't intelligent, but they are an early path towards true artificial intelligence

[Yuli Ban]

VGPNN: Diverse Generation from a Single Video Made Possible

Most advanced video generation and manipulation methods train on a large collection of videos. As such, they are restricted to the types of video dynamics they train on. To overcome this limitation, GANs trained on a single video were recently proposed. While these provide more flexibility to a wide variety of video dynamics, they require days to train on a single tiny input video, rendering them impractical.

In this paper we present a fast and practical method for video generation and manipulation from a single natural video, which generates diverse high-quality video outputs within seconds (for benchmark videos). Our method can be further applied to full-HD video clips within minutes. Our approach is inspired by a recent advanced patch-nearest-neighbor based approach [Granot et al., 2021], which was shown to significantly outperform single-image GANs, both in run-time and in visual quality.

Here we generalize this approach from images to videos, by casting classical space-time patch-based methods as a new generative video model. We adapt the generative image patch nearest neighbor approach to efficiently cope with the huge number of space-time patches in a single video. Our method generates more realistic and higher quality results than single-video GANs (confirmed by quantitative and qualitative evaluations). Moreover, it is disproportionally faster (runtime reduced from several days to seconds). Other than diverse video generation, we demonstrate several other challenging video applications, including spatio-temporal video retargeting (e.g., video extension & video summarization), video structural analogies and conditional video-inpainting.

https://nivha.github.io/vgpnn/vids/anal ... es_exp.mp4

[Yuli Ban]

[Yuli Ban]

[Yuli Ban]

Baidu Releases PLATO-XL: World’s First 11 Billion Parameter Pre-Trained Dialogue Generation Model

It has always been a great challenge for AI bots to conduct coherent, informative, and engaging conversations as human beings. For robots to serve as emotional companions or intelligent assistants it is essential that they develop high-quality open-domain dialogue systems. As pre-training technology further promotes models’ ability to learn from large-scale unannotated data, mainstream research is focusing on making more efficient and full use of massive data to improve open-domain dialogue systems. To this end, Baidu releases the PLATO-XL with up to 11 billion parameters, achieving new breakthroughs in Chinese and English conversations.

· PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation

· https://arxiv.org/abs/2109.09519
In recent years we have witnessed constant progress in the field of open-domain conversation, from Google’s Meena and Facebook’s Blender to Baidu’s PLATO. In DSTC-9, the top dialog system technology challenge, Baidu PLATO-2 broke a record by winning the first place in five different dialogue tasks.

Now Baidu PLATO-2 has been upgraded to PLATO-XL. Over ten billion parameters make it the world’s largest Chinese and English dialogue generation model. Achieving superior performance in open-domain conversation, PLATO-XL raises our expectation of what hundred-billion or even trillion parameter dialogue models could do.

[Yuli Ban]

Artificial intelligence might eventually write this article
GPT-3 is turning AI into idea generators and writers

I hope my headline is an overstatement, purely for job purposes, but in this week’s Vergecast artificial intelligence episode, we explore the world of large language models and how they might be used to produce AI-generated text in the future. Maybe it’ll give writers ideas for the next major franchise series, or write full blog posts, or, at the very least, fill up websites with copy that’s too arduous for humans to do.

Among the people we speak to is Nick Walton, the cofounder and CEO of Latitude, which makes the game AI Dungeon, which creates a plot in the game around what you put into it. (That’s how Walton ended up in a band of traveling goblins — you’ll just have to listen to understand how that makes sense!) We also chat with Samanyou Garg, founder of Writesonic, a company that offers various writing tools powered by AI. The company can even have AI write a blog post — I’m shaking! But really.

[Yuli Ban]

We are excited to introduce the DeepSpeed- and Megatron-powered Megatron-Turing Natural Language Generation model (MT-NLG), the largest and the most powerful monolithic transformer language model trained to date, with 530 billion parameters. It is the result of a joint effort between Microsoft and NVIDIA to advance the state of the art in AI for natural language generation.
As the successor to Turing NLG 17B and Megatron-LM, MT-NLG has 3x the number of parameters compared to the existing largest model of this type and demonstrates unmatched accuracy in a broad set of natural language tasks such as:

• Completion prediction
• Reading comprehension
• Commonsense reasoning
• Natural language inferences
• Word sense disambiguation

The 105-layer, transformer-based MT-NLG improved upon the prior state-of-the-art models in zero-, one-, and few-shot settings and set the new standard for large-scale language models in both model scale and quality.

[Yuli Ban]

[Yuli Ban]

[bretbernhoft]

In some senses it doesn't really matter if something is "real" or "fake"; there is likely someone (who fully knows what "it is") who is willing to buy it regardless. I mean to say that there are already whole communities of folks who find immeasurable value in digital assets. Sitting here, I wonder what the relationships between (so-called) fake digital assets and cryptocurrencies are?

[Yuli Ban]

This whole area is starting to really pick up in terms of how many people are following it. I've been watching generative AI develop for years now, ever since my epiphany on it back in 2017, and back then I could only scarcely fathom how far it would go in just four years. In some ways, it's not gone quite as far as I hoped, but that's only in specifics— I'm still awaiting the AI-generated comic book I predicted. But in others, just holy crap. Text-to-image synthesis alone would've blown my mind back then! Gonna post a whole mess of those outputs people have made.

[Yuli Ban]

In some senses it doesn't really matter if something is "real" or "fake"; there is likely someone (who fully knows what "it is") who is willing to buy it regardless. I mean to say that there are already whole communities of folks who find immeasurable value in digital assets. Sitting here, I wonder what the relationships between (so-called) fake digital assets and cryptocurrencies are?

It matters only in so far that people "trust" what they see is real. If everything people see is fake, they'll still trust an alternative. It just comes down to what people choose to trust in that situation. And clearly that could be exploited.

[Yuli Ban]

"ADOP: Approximate Differentiable One-Pixel Point Rendering", Rückert et al 2021

We present a novel point-based, differentiable neural rendering pipeline for scene refinement and novel view synthesis. The input are an initial estimate of the point cloud and the camera parameters. The output are synthesized images from arbitrary camera poses. The point cloud rendering is performed by a differentiable renderer using multi-resolution one-pixel point rasterization. Spatial gradients of the discrete rasterization are approximated by the novel concept of ghost geometry. After rendering, the neural image pyramid is passed through a deep neural network for shading calculations and hole-filling. A differentiable, physically-based tonemapper then converts the intermediate output to the target image. Since all stages of the pipeline are differentiable, we optimize all of the scene's parameters i.e. camera model, camera pose, point position, point color, environment map, rendering network weights, vignetting, camera response function, per image exposure, and per image white balance. We show that our system is able to synthesize sharper and more consistent novel views than existing approaches because the initial reconstruction is refined during training. The efficient one-pixel point rasterization allows us to use arbitrary camera models and display scenes with well over 100M points in real time.

[Yuli Ban]

[Yuli Ban]

In 2020, a group of robbers in the United Arab Emirates used deepfake technology to imitate the voice of a bank executive to deceive a manager into handing them over money. The government of the UAE has requested the United States for help in tracing the approximately $400,000 that went into the suspects' accounts.

This hasn't been the first time AI has been used for nefarious purposes. In 2019, criminals deepfaked an executive's voice in an attempt to steal $243,000 according to the Wall Street Journal.

[wjfox]

[Yuli Ban]

[Yuli Ban]

What is synthetic media, and how is it distinguished from digital human technology?

Synthetic media has become synonymous with the aspects of manipulation, particularly in deep fakes (and boy, will we come on to deep fakes). That has largely been down to how the technology has hit the public consciousness, driven by concerns over fake news and the genuine concern that people may use it to manipulate our perceptions of reality.

But synthetic media as a category and array of technologies is much more than just deep fakes. There are interesting, creative applications of synthetic media in popular culture and commercial settings.

There’s also an interesting intersection between synthetic media and digital humans, all of which we’ll unpack below.

What is synthetic media?
Synthetic media is an all-encompassing term for the artificial creation or modification of media by “machines” – particularly programs that rely on artificial intelligence and machine learning.

In other words, it’s media that is produced by technology. For this reason, you might also hear synthetic media called “AI-generated media”.

Some types of synthetic media today include AI-written music, text generation, imagery and video, voice synthesis and more. The field is ever expanding as synthetic media companies aim to disrupt more and more parts of traditional media, making new things easier to create.