It's a terrible analogy because the entire point of ML systems is to generalize well to new data, not to reproduce the original data as accurate as possible with a space/time tradeoff.
I don't think you can describe the math in this context as "generalize well to new data."
ChatGPT certainly can't generate new data. It's not gonna correctly tell you today who won the World Series in 2030. It's not going to write a poem in the style of someone who hasn't been born yet.
But it can interpolate between and through a bunch of existing data that's on the web to produce novel mixes of it. I find the "blurring those things together" analogy pretty compelling there, in the same way that blurring or JPEG-compressing something isn't going to give you a picture of a new event but it might change what you appear to see in the data you already had.
(Obviously it's not exactly the same, that's why it's an analogy and not a definition. As an analogy, it works much better if you ignore much of what you know about the implementation details of both of them. It's not trying to teach someone how to build it, but to teach a lay person how to think about the output.)
It absolutely can generate new data, it does so all the time. If you are claiming otherwise I think we need a more formal definition of what you mean by new data.
Are you suggesting because it can't predict the future it can't generate novel data?
It's not just the future, though the examples I gave were future oriented.
But it's all very interpolation/summarization-focused.
A "song lyrics in the style of Taylor Swift" isn't an actual song by Taylor Swift.
A summary of the history of Texas isn't actually vetted by any historian to ensure accuracy.
The answer to a math problem may not be correct.
To me, those things don't qualify as "new data." They aren't suitable for future training as-is. Sometimes for a simple reason: they aren't facts, using the dictionary "facts and statistics collected together for reference or analysis" definition of data. So very simply "not new data."
Sometimes in a blurrier way - the song lyrics, for instance, could be touching, or poignant, or "true" in a Keats sense[0] - but if the internet gets full of GPT-dreams and future models are trained on that, you could slide down further and further into an uncanny valley, especially since most of the time you don't get one of those amazing poignant ones. Most of the time I've gotten something bland.
[0] "What the imagination seizes as beauty must be truth"
One way to think about prompting is as a conditional probability distribution. There is a particular song by Taylor Swift or the set of all songs by Taylor Swift but ChatGPT is particularly talented at sampling the "set of all songs in the style of Taylor Swift".
One of the worst problems in the "Expert Systems" age of A.I. was reasoning over uncertainty, for instance this system
had a half-baked approach that worked well enough for a particular range of medical diagnosis. In general it is an awful problem because it involves sampling over a joint probability distribution. If you have 1000 variables you have to sample a 1000-dimensional space, to do it the brute force way you'd have sample the data in an outrageous number of hypercubes.
Insofar as machine learning is successful it is that we have algorithms that take a comparatively sparse sample and make a good guess of what the joint p.d. is. The success of deep learning is particularly miraculous in that respect.
The thing is that generalization is good enough to make people squee and not notice that the output is wrong but not good enough to get the right answer.
If it were going to produce ‘explainable’ correct answers for most of what it does that would be a matter of looking up the original sources to make sure they really say what it thinks they do. I mean, I can say, “there’s this paper that backs up my point” but I have to go look it up to get the exact citation at the very least.
There is definitely a misconception about how to use a tool like ChatGPT.
If you give it an analytic prompt like "turn this baseball box score into an entertaining outline" it will reliably act as a translator because all of the facts about the game are contained in the prompt.
If you give it a synthetic prompt like "give me quotes from the broadcasters" it will reliably acts as a synthesizer because none of the facts of the transcript are in the prompt.
This ability to perform as a synthesizer is what you are identifying here as "good enough to make people squee and not notice that the output is wrong but not good enough to get the right answer", which is correct, but sometimes fiction is useful!
If all web pages were embedded in ChatGPT's 1536 dimensional vector space and used for analytic augmentation then a tool would more reliably be able to translate a given prompt. The UI could also display the URLs of the nearest-neighbor source material was used to augment the prompt. That seems to be what Bing/Edge has in store.
That's a touch beyond state of the art but we might get there.
If there was one big problem w/ today's LLMs it is that the attention window is too short to hold a "complete" document. I can put the headline of an HN submission through BERT and expect BERT to capture it but there is (as of yet) no way to cut up a document up into 512 (BERT) or 4096 (ChatGPT) token slices and then mash those embeddings together to make an embedding that can do all the things the model is trained to do on a smaller data set. I'm sure we will see larger models, but it seems a scalable embedding that grows with the input text would be necessary to move to the next level.
It's built with Supabase/Postgres, and consists of several key parts:
Parsing the Supabase docs into sections.
Creating embeddings for each section using OpenAI's embeddings API.
Storing the embeddings in Postgres using the pgvector extension.
Getting a user's question.
Query the Postgres database for the most relevant documents related to the question.
Inject these documents as context for GPT-3 to reference in its answer.
Streaming the results back to the user in realtime.
The same thing could be done with search engine results and from recent demos it looks like this is the kind of analytic augmentation that MS and OpenAI have added to Bing.
