The Singularity - Official Thread
Re: The Singularity - Official Thread
We will know years in advance before a new type of computer is going to be mass produced. Computer chips of 2026 will be produced in fabs which are already under construction and we knew in 2020 they were going to be constructed. There is no revolutionary new computer design coming before the 2030s. It's not impossible that one day we will see computers jump 1000x in performance, but for now the trend for personal computers is nothing of the sort. Supercomputer GPUs get 2x faster every 2 years, but they also get more expensive. Desktop CPUs and GPUs get twice as fast after 4 years at constant price (GTX 1060 vs RTX 3060, Ryzen 1800X vs Ryzen 5900X, i5-8400 vs i5-12400). But overall I agree with him.
Global economy doubles in product every 15-20 years. Computer performance at a constant price doubles nowadays every 4 years on average. Livestock-as-food will globally stop being a thing by ~2050 (precision fermentation and more). Human stupidity, pride and depravity are the biggest problems of our world.
Re: The Singularity - Official Thread
OpenAI hinting that they may have made an AI with consciousness.
A reddit user said "A lot of recent statements from AGI CEOs such as Altman and the recent developments makes us feel like they’re trying to break some news gently…"
I recall Sam Altman saying
and an OpenAI employee saying 5% chance of AGI in 2022
As much as I am a schizo for AI I don't want to try and read into a company dropping hints lmao.
Maybe this could be to do with proto AGI.
A reddit user said "A lot of recent statements from AGI CEOs such as Altman and the recent developments makes us feel like they’re trying to break some news gently…"
I recall Sam Altman saying
and an OpenAI employee saying 5% chance of AGI in 2022
As much as I am a schizo for AI I don't want to try and read into a company dropping hints lmao.
Maybe this could be to do with proto AGI.
Re: The Singularity - Official Thread
https://blogs.microsoft.com/ai-for-busi ... echnology/
A reddit users comment
""One important approach to advancing model capabilities is the training of expert models with subtasks using an ensemble method, named Mixture of Experts (MoE). The MoE architecture also preserves sublinear computation with respect to model parameters that provides a promising path to improve model quality via scaling out trillions of parameters without increasing training cost. We have also developed MoE models to our pretrained model family accelerated with DeepSpeed. (DeepSpeed details can be found below.)
Large models with billions or even trillions of parameters require a host of optimization and parallelization strategies to be trained efficiently. So, we developed a set of optimization techniques, such as the Zero Redundancy Optimizer (ZeRO), and open sourced them into a PyTorch acceleration library called DeepSpeed since 2020. DeepSpeed continues to trailblaze with an extreme focus on speed, scale, efficiency, usability and democratization of large-scale model training and inferencing.
ZeRO-Infinity is another feature in DeepSpeed. It breaks the GPU memory wall to enable training models with trillions of parameters by offloading parameters and state to the NVMe disk storage, enabling you to train large models on a single GPU or a single node with a few GPUs. For example, you can fine-tune a GPT-3-sized model on a single node thanks to ZeRO-Infinity to create the ability to train very large models on more modest hardware. Using DeepSpeed optimizations our internal production workloads have seen anywhere from 2x to 20x bigger model scale and fast training for large models.
Our commitment to sharing the key building blocks for these models as a platform via open source remains strong. We continuously innovate in DeepSpeed, ONNX Runtime and the training recipes in Azure Machine Learning to push the boundaries of efficiency to train and serve the models and increase scale of models to trillions of parameters. We’ll continue to democratize the access and ability to train or adapt ever-larger base models and deploy these models to the cloud."
This blogpost seems like a dry business report on the surface, but it outlines many aspects of Microsoft's strategy moving forward, including what based on the above quotes, looks like a multi-trillion parameter language model which will likely have multi-modal capabilities.
If scaling to trillions of parameters results in further emergent abilities, we will become aware of it soon.""
A reddit users comment
""One important approach to advancing model capabilities is the training of expert models with subtasks using an ensemble method, named Mixture of Experts (MoE). The MoE architecture also preserves sublinear computation with respect to model parameters that provides a promising path to improve model quality via scaling out trillions of parameters without increasing training cost. We have also developed MoE models to our pretrained model family accelerated with DeepSpeed. (DeepSpeed details can be found below.)
Large models with billions or even trillions of parameters require a host of optimization and parallelization strategies to be trained efficiently. So, we developed a set of optimization techniques, such as the Zero Redundancy Optimizer (ZeRO), and open sourced them into a PyTorch acceleration library called DeepSpeed since 2020. DeepSpeed continues to trailblaze with an extreme focus on speed, scale, efficiency, usability and democratization of large-scale model training and inferencing.
ZeRO-Infinity is another feature in DeepSpeed. It breaks the GPU memory wall to enable training models with trillions of parameters by offloading parameters and state to the NVMe disk storage, enabling you to train large models on a single GPU or a single node with a few GPUs. For example, you can fine-tune a GPT-3-sized model on a single node thanks to ZeRO-Infinity to create the ability to train very large models on more modest hardware. Using DeepSpeed optimizations our internal production workloads have seen anywhere from 2x to 20x bigger model scale and fast training for large models.
Our commitment to sharing the key building blocks for these models as a platform via open source remains strong. We continuously innovate in DeepSpeed, ONNX Runtime and the training recipes in Azure Machine Learning to push the boundaries of efficiency to train and serve the models and increase scale of models to trillions of parameters. We’ll continue to democratize the access and ability to train or adapt ever-larger base models and deploy these models to the cloud."
This blogpost seems like a dry business report on the surface, but it outlines many aspects of Microsoft's strategy moving forward, including what based on the above quotes, looks like a multi-trillion parameter language model which will likely have multi-modal capabilities.
If scaling to trillions of parameters results in further emergent abilities, we will become aware of it soon.""
Re: The Singularity - Official Thread
Findings: No Singularity Imminent
There is no upcoming technological singularity, and complexity in our lives will likely begin decreasing “sometime in the not too distant future,” says Theodore Modis, a scientist and strategic analyst specializing in the study of growth and cycles in a variety of contexts.
Modis’s article “Links between Entropy, Complexity, and the Technological Singularity” in the journal Technological Forecasting & Social Change (March 2022) corroborates 20 years of research that has enabled him “to establish a simple mathematical relationship between entropy and complexity,” he stated in an email. “This relationship provides general guidelines on how complexity relates to entropy, for example complexity is higher when entropy grows faster,” he writes.
Limited-time access to the article is free on Elsevier’s website, or readers may download a preprint here.
Read more: http://www.aaiforesight.com/newsletter/ ... y-and-more
There is no upcoming technological singularity, and complexity in our lives will likely begin decreasing “sometime in the not too distant future,” says Theodore Modis, a scientist and strategic analyst specializing in the study of growth and cycles in a variety of contexts.
Modis’s article “Links between Entropy, Complexity, and the Technological Singularity” in the journal Technological Forecasting & Social Change (March 2022) corroborates 20 years of research that has enabled him “to establish a simple mathematical relationship between entropy and complexity,” he stated in an email. “This relationship provides general guidelines on how complexity relates to entropy, for example complexity is higher when entropy grows faster,” he writes.
Limited-time access to the article is free on Elsevier’s website, or readers may download a preprint here.
Read more: http://www.aaiforesight.com/newsletter/ ... y-and-more
-
funkervogt
- Posts: 1192
- Joined: Mon May 17, 2021 3:03 pm
Re: The Singularity - Official Thread
Wow, that's scary.wjfox wrote: ↑Fri Feb 11, 2022 3:13 pm Findings: No Singularity Imminent
There is no upcoming technological singularity, and complexity in our lives will likely begin decreasing “sometime in the not too distant future,” says Theodore Modis, a scientist and strategic analyst specializing in the study of growth and cycles in a variety of contexts.
Modis’s article “Links between Entropy, Complexity, and the Technological Singularity” in the journal Technological Forecasting & Social Change (March 2022) corroborates 20 years of research that has enabled him “to establish a simple mathematical relationship between entropy and complexity,” he stated in an email. “This relationship provides general guidelines on how complexity relates to entropy, for example complexity is higher when entropy grows faster,” he writes.
Limited-time access to the article is free on Elsevier’s website, or readers may download a preprint here.
Read more: http://www.aaiforesight.com/newsletter/ ... y-and-more
Hmm. This claim could be challenged. The last of the 28 milestones is "Internet, human genome sequenced."In any case complexity, as determined by the 28 milestones, has reached a maximum and now begins on the declining slope of its bell-shaped pattern. It is a direct consequence of having described the accumulation of entropy by a natural-growth (logistic) pattern, which so far seems to hold as there haven’t been any ―milestones in the last 25 years. There have been many small ones but nothing like the Internet, DNA, or nuclear energy. The idea that our world’s complexity will decrease in the future may seem difficult to accept but such a unimodal pattern(namely low at the beginning and at the end but high in between, not unlike the normal—Gaussian—distribution3)is commonplace in everyday life.
The internet became common in the late 90s and the human genome was first sequenced in 2001 or 2003 (depending on how you look at it). Since then, there have been other, very impactful tech milestones:
-Invention of smartphones and their rapid proliferation
-Invention of social media
-Can anyone think of more?
Additionally, we might look back on the advent of cryptocurrency as being so impactful that it deserves inclusion on the milestone list. We need more time to see how it pans out.
The rise of affordable, high-quality VR technology and of machines that can pass the Turing Test will probably happen in the next ten years, and will also probably be so impactful that they will count as milestones on the list.
Re: The Singularity - Official Thread
He didn't even consider the real possibility of A.I. taking the reins from us in terms of generating breakthroughs at least on par with the Internet and other equally paradigm-shifting revolutions.
It's a flawed analysis, in my view. It's basically a modern rehashing of "going more than 45 mph will asphyxiate us" quipped by some 19th century dude.
It's a flawed analysis, in my view. It's basically a modern rehashing of "going more than 45 mph will asphyxiate us" quipped by some 19th century dude.
To know is essentially the same as not knowing. The only thing that occurs is the rearrangement of atoms in your brain.
Re: The Singularity - Official Thread
And remember my friend, future events such as these will affect you in the future