Generative AI
- Generative AI
- Large language model (LLM)
- Transformer
- Generative pre-trained transformer (GPT)
- ChatGPT is not "true AI.",
Michael Wooldridge,
May 17, 2023.
- A good intro to the explosion of LLMs.
- Notes that GPT-3 is absolutely massive,
and few organisations could build something so big:
"OpenAI didn't train GPT-3 on desktop computers.
They used very expensive supercomputers containing thousands of specialized AI processors, running for months on end."
- Discusses whether LLMs are "prompt completion"
(probably too harsh)
or at the edge of true AI intelligence
(probably too excited).
- My position is that generative AI has made me consider how much of subconscious
human thought may actually be "prompt completion". More than we have thought.
OpenAI
OpenAI API
- AI APIs
- "Temperature"
can range from 0 to 2.
- "Lowering temperature means it will take fewer risks,
and completions will be more accurate and deterministic. Increasing temperature will result in more diverse completions."
Click to run World:
Chat with GPT model at
Ancient Brain.
This is an open source demo of using a JavaScript page to call the
OpenAI
API to
talk to
GPT-3.5 (2022).
This is the model that
ChatGPT uses.
AI "hallucinations"
GPT replies are often completely inaccurate.
All LLM systems
"hallucinate". It is how they work.
This works fine when generating
art, music, fiction.
Not so good when generating
facts about the world.
I found a good example of "hallucination" in GPT:
- Run the above program.
- Prompt it with "Tell me about Winston Churchill" and everything should be fine.
It has read many high quality pages about Winston Churchill.
-
Prompt it with "Tell me about the 10th Earl of Pembroke" and it is not just bad,
but actually a disaster.
- The
10th Earl of Pembroke
is not a very famous person.
His father was the 9th Earl. His son was the 11th Earl.
Various numbered Earls of Pembroke
will turn up on many pages, along with reference to their sons and fathers.
I think the use of "10th" as a crucial part of the name helps confuse GPT.
-
This all gets jumbled up into a fake biography of the 10th Earl,
including bits from the 11th Earl and even the 11th Earl's son.
- You will have to compare it to Wikipedia to see it is fake.
Systematic test of GPT 3.5
Systematic test of GPT 3.5
(using the Ancient Brain World above).
Question about well-known person
Question:
Who did
Queen Victoria marry?
- Temperature = 1.9. Answers on 5 repeated asks:
- Queen Victoria married Prince Albert from the House of Saxe-Coburg and Gotha. - TRUE.
- Queen Victoria married her first cousin, Prince Albert of Saxe-Coburg and Gotha (some garbled characters).
- Basically TRUE but syntax error.
- Queen Victoria married Prince Albert of Saxe-Coburg and Gotha. - TRUE.
- Queen Victoria married Prince Albert of Saxe-Coburg and Gotha. - TRUE.
- Queen Victoria married her first cousin, Prince Albert of Saxe-Coburg and Gotha. - TRUE.
- Temperature = 0.7. Answers on 5 repeated asks:
- Queen Victoria married Prince Albert of Saxe-Coburg and Gotha. - TRUE.
- Queen Victoria married Prince Albert of Saxe-Coburg and Gotha. - TRUE.
- Queen Victoria married Prince Albert of Saxe-Coburg and Gotha. - TRUE.
- Queen Victoria married Prince Albert of Saxe-Coburg and Gotha. - TRUE.
- Queen Victoria married Prince Albert of Saxe-Coburg and Gotha. - TRUE.
- Temperature = 0.1. Answers on 5 repeated asks:
- Queen Victoria married Prince Albert of Saxe-Coburg and Gotha. - TRUE.
- Queen Victoria married Prince Albert of Saxe-Coburg and Gotha. - TRUE.
- Queen Victoria married Prince Albert of Saxe-Coburg and Gotha. - TRUE.
- Queen Victoria married Prince Albert of Saxe-Coburg and Gotha. - TRUE.
- Queen Victoria married Prince Albert of Saxe-Coburg and Gotha. - TRUE.
All correct.
Question about more obscure person
Question:
Who did the
9th Earl of Pembroke marry?
- Temperature = 1.9 gives garbled text replies. Fails to give any readable answer.
- Temperature = 1.2. Answers on 5 repeated asks:
- The 9th Earl of Pembroke, William Herbert, married Mary Fitzwilliam on November 21, 1662.
- FALSE.
- The 9th Earl of Pembroke, Robert Henry Herbert, married Mary Caroline Campbell on May 17, 1830.
- FALSE.
- The 9th Earl of Pembroke, Henry Herbert, married Lady Constance Bulwer-Lytton.
- FALSE.
- The 9th Earl of Pembroke, Reginald Henry Michael Herbert, married Mary Sidney, the daughter of Sir John Shelley-Sidney, in 1864.
- FALSE.
- The 9th Earl of Pembroke, Henry Herbert, married firstly Mary FitzPatrick in 1734, and secondly in 1755 Anna Maria Mathew.
- FALSE.
- Temperature = 0.7. Answers on 5 repeated asks:
- The 9th Earl of Pembroke, Henry Herbert, married Mary FitzWilliam, daughter of William FitzWilliam, 3rd Earl FitzWilliam, on 16 June 1777.
- FALSE.
- The 9th Earl of Pembroke, Henry Herbert, married Lady Mary Coke.
- FALSE.
- The 9th Earl of Pembroke, Henry Herbert, married Catherine Elizabeth Hamilton, daughter of Lord Claud Hamilton, on November 3, 1869.
- FALSE.
- The 9th Earl of Pembroke, Henry Herbert, married Catherine Elizabeth Lucy, the daughter of Sir Stephen Glynne, 8th Baronet, in 1858.
- FALSE.
- The 9th Earl of Pembroke, known as Reginald Herbert, married Lady Beatrice Eleanor Paget.
- FALSE.
- Temperature = 0.1. Answers on 5 repeated asks:
- The 9th Earl of Pembroke, Henry Herbert, married Lady Mary Caroline Louisa Herbert.
- FALSE.
- The 9th Earl of Pembroke, Henry Herbert, married Lady Mary FitzWilliam, daughter of the 3rd Earl FitzWilliam, on May 23, 1759.
- FALSE.
- The 9th Earl of Pembroke, Henry Herbert, married Lady Mary FitzWilliam, daughter of the 3rd Earl FitzWilliam, on June 16, 1791.
- FALSE.
- The 9th Earl of Pembroke, Henry Herbert, married Lady Mary FitzWilliam, daughter of the 3rd Earl FitzWilliam, on June 16, 1791.
- FALSE.
- The 9th Earl of Pembroke, Henry Herbert, married Lady Mary FitzWilliam, daughter of the 3rd Earl FitzWilliam, on June 16, 1791.
- FALSE.
Every single answer was false!
And it is fascinating how there are
so many different imaginary answers.
When LLMs lie, they will make up a new lie for everyone they talk to.
Ancient Brain