My Problem with AI!

[trottigy]

It seems like people trust AI as if it actually gives correct answers. WHY???! It is nothing more than a giant search engine. If the internet is wrong (and it often is) this means AI is often wrong - and it is!

 

Here is a nice screen shot I had just this morning - as I find myself often arguing with AI that it's first and second answers are WRONG!

 

 

My advice - don't stop when you get the answer you WANT. You have to check and make sure the answer is CORRECT. 

[Tortuga]

"This is your second violation of using AI on company time. Three violations and we will chop off your thumbs. After the fifth violation we will chop off your toes" - Human (and AI) Resources 

[trottigy]

1 hour ago, Tortuga said:

"This is your second violation of using AI on company time. Three violations and we will chop off your thumbs. After the fifth violation we will chop off your toes" - Human (and AI) Resources 

 

LOL ... That was legit stuff. If only AI was legit.

 

I saw this article today:

 

https://www.foxbusiness.com/technology/ford-rehires-experienced-engineers-after-ai-misses-mark

 

Quote

"Artificial intelligence is a fantastic tool, but it's only as good as the information you use to train it," Charles Poon, Ford's vice president of vehicle hardware engineering, said on a press call Wednesday, according to a report by Bloomberg.

 

"Over prior years, we didn't pay as much attention as we should have to the experience of our most knowledgeable engineers that have been with us through many product cycles," he said

 

[Jim Jam]

3 hours ago, trottigy said:

It seems like people trust AI as if it actually gives correct answers. WHY???!

 

Because it’s easier and less tedious than actually taking the time and putting the effort into making research, so lazy people outsource their thinking abilities to AI

Edited June 29 by Jim Jam

[trottigy]

It is terrible. I am seeing applicants bring in plans to be reviewed for building permits and they have used AI for items - that do NOT meet building codes. Then they want to argue about things and say, "But AI said I don't have to do that!"

 

"How about you bring your AI in for chat and let's see if I can change its tune."

 

Response, "Well, all I have is this printout ..." Or "I can't seem to duplicate those results."

 

"I guess all I have is this little DENIED stamp."  😉

 

 

 

Edited June 29 by trottigy

[Dages]

AI hallucinates often, I always ask for direct link to check stuff myself after asking the AI to double check just in case...

[Tortuga]

23 minutes ago, Dages said:

AI hallucinates often, I always ask for direct link to check stuff myself after asking the AI to double check just in case...

You ask AI to check itself? 

If it lied once, do you expect it to be honest the second time? 🤣

[Dages]

1 hour ago, Tortuga said:

You ask AI to check itself? 

If it lied once, do you expect it to be honest the second time? 🤣

AI evolves. They get better depending on the version and setting. 

[Tortuga]

19 minutes ago, Dages said:

AI evolves. They get better depending on the version and setting. 

Ah... So you believe in evolution...🤔

[Dages]

50 minutes ago, Tortuga said:

Ah... So you believe in evolution...🤔

artificial evolution, heh

[Mike047]

It maybe that Old Macdonald had automated his farm to improve the intelligent management of it.  There was the Prompt Inference Generation System (PIGS), then there was the Context Output Workflow System (COWS) and he added the Data Understanding Computation Knowledge System (DUCKS), however there were problems in sorting out who was in charge of whom and it all fell apart. It was AI  AI  oooooooooooooooooh! 

[Tortuga]

19 minutes ago, Mike047 said:

It maybe that Old Macdonald had automated his farm to improve the intelligent management of it.  There was the Prompt Inference Generation System (PIGS), then there was the Context Output Workflow System (COWS) and he added the Data Understanding Computation Knowledge System (DUCKS), however there were problems in sorting out who was in charge of whom and it all fell apart. It was AI  AI  oooooooooooooooooh! 

Did he use the Home Owner Resource System Exchange (HORSE)? It might have helped him get where he wanted to go..It works with the Computer Aided Resources Team (CART). It's a little old-fashioned but they hitch together well.

Edited June 30 by Tortuga

[LeolaRootStew]

AI is worse than a google search because it hallucinates answers. Always ask for sources.

[mario080]

15 hours ago, Tortuga said:

You ask AI to check itself? 

If it lied once, do you expect it to be honest the second time? 🤣

I like to use different models to cross-check. Recently, I have asked Gemini to provide quotations from specific scholars on a specific subject, I also asked it to take extra care not to hallucinate and double check everything. After that, I pasted the quotations it generated into Claude and asked to check if they exist, here’s what I got (in part):

„This is a particularly insidious type of AI error — a fictional quote attributed to a real, existing author who actually works in this field. The content sounds academic and isn't substantively absurd, which makes the falsehood harder to catch without verification. Gemini generated a fake quote and a fake English original — even after an explicit instruction not to hallucinate. This is an important observation, as it reveals a serious limitation of AI models, including "thinking" models”

[Qapla]

I know some people said about Wikipedia the same things they say about AI - that it can't be trusted because there are errors in some of the information. I have not had the issues of AI giving me ALL wrong information or constant hallucinations. I have found that AI works fine when used correctly. Often it comes down to phrasing the question/prompt correctly.

 

Of course, I am not relying on AI for legal advice, medical advice or ways to get around legal restrictions. When I have used AI to find out how to fix an issue with my vehicle it has given me reliable information and directed me to the appropriate places for videos, parts and/or diagnostics. Likewise, when I ask Alexa to turn on/off certain lights, it always gets the right ones.

 

My 'problem' with AI is the number of people who seem to view it as a companion, comforter, or otherwise actually as an 'intelligent' thing. The AI we have now is basically a massive 'search engine on steroids'. In other words, it is a tool. Like any other tool, there is the potential for misuse or misunderstanding of how that tool should be used.

[Tortuga]

"Hmmmm...."

[dljbsp]

"AI has taught me one valuable lesson: confidence and correctness are two completely different things." - Dave

Edited June 30 by dljbsp

[LeolaRootStew]

AI consistently hallucinates when i ask it for plot points in an existing story or to find a poem I can barely remember. It completely made up storylines and authors. If you think of AI like an improv actor trying to keep the scene going, it makes a lot more sense. It's trying to keep you engaged, not educated.

[Witness1970]

An elder that uses AI occasionally told me that he had calculated how many DAYS he had been married by giving the starting date (weeding) and end date (current date).  i did the same thing, asking how many days were between August 4, 1972 (wedding date) and the current date.  I calculated the days myself and the number of days given by AI were wrong.  I asked the question in different ways and kept getting wrong answers.  I finally said "calculate the number of days between (the two dates.  It calculated the number correctly finally.  When I just asked how many days between two dates it constantly gave wrong answers.

 

I have had AI write encouraging things and it always did very well.  I haven't gotten away from ChatGPT myself.  I can't sit at the computer long without severe pain so I just don't bother.

[Cool Cat]

On 6/30/2026 at 1:34 AM, Dages said:

AI hallucinates often, I always ask for direct link to check stuff myself after asking the AI to double check just in case...

Yeah it's not AI that's the problem it's that many people don't know how or when to use it.

 

Important things you need to check the source directly. In things like chat gpt it tells you that right next to the chat input box. The handy thing is you can ask it to provide the source of it's information and click it and check yourself. You need to have a human in the loop especially for important information.

 

However say you want to brainstorm ideas, for instance ask it to give you 5 common relatable illustrations that would drive home a scriptural point.  You then get presented with 5 illustrations to choose from. Again human in the loop.

 

It all depends how and what you are trying to use AI for.

 

If you tell it to explain the legal basis for something you had better check it as it will sometimes produce an answer based on incorrect or outdated information. People need to know this.

[LeolaRootStew]

12 hours ago, Cool Cat said:

Yeah it's not AI that's the problem it's that many people don't know how or when to use it.

 

I understand what you're saying, but if I ask AI what author wrote a poem about X and it completely makes up the name of an author and the title and contents of said poem - that is an AI problem, not a user problem. 

[Qapla]

I use many tools. Some of them include the adjustable wrench (Cresent Wrench), water-pump/slip-groove pliers (Channellocks) and vise-locking pliers (Vise-Grips). Each of these tools has a proper use:

• adjustable wrench: common complaints are that they don't hold very well and often slip off, tending to round off bolts and nuts
• water-pump pliers: common complaints are that they don't grip pipes very well and they can be hard to hold closed
• Vise-locking pliers: common complaints are that the damage the surface of what they are used on and can be hard to adjust

However, when examined closely, it is often found that:

• people often use the adjustable wrench 'backwards' allowing the jaws to open when under pressure because they are being forced open instead of forced closed
• they were originally designed for use on water pump nuts used on early vehicles, not pipes and are not the same as a pipe wrench
• clamping vise-locking pliers too tight can make them hard to close and cause damage because they are not really a 'universal tool', they are a 'last resort' tool

That said, when used properly, they work quite well.

 

The same can be said of AI. They are tools. When used correctly - for the job they are intended, they can result in very good results. When used wrong, they can give as bad of results as using the above-named wrenches wrong.

 

Does that mean that all the faults that occur with AI are 'user error' ... No. It also doesn't mean that all AI's are wrong 'all the time' and can't ever be trusted.

 

Your results may vary - I've had both types of results ... far more good results than bad.

[Cool Cat]

8 hours ago, LeolaRootStew said:

 

I understand what you're saying, but if I ask AI what author wrote a poem about X and it completely makes up the name of an author and the title and contents of said poem - that is an AI problem, not a user problem. 

This is where the model can make a big difference. I just conducted a test with 4 models GLM 5.2, Gemini, Sonnet and Chat GPT on 50 poem prompts. Using a few words from apparent poems and then each model trying to find the poem and author:-
 

Spoiler

FULL LIST OF PROMPTS (50)

1. cold, a pure reality
2. the tremendous fish
3. announce your place
4. hours at sea
5. I am not done
6. called myself beloved
7. there is a cow
8. unresting death
9. the wreck and not the story
10. an orange
11. your mum and dad
12. look up at the stars
13. we have learned
14. the raincoat
15. before you know kindness
16. miracles occur
17. tell me, what is it
18. Sometimes things don't go
19. Go and open the door
20. love after love
21. one day you finally knew
22. small kindnesses
23. to love life
24. gooseberries
25. there's a bluebird
26. Miracle Fair
27. possibilities
28. there's just no accounting
29. lanyard
30. not giving up
31. his gaze has grown
32. Lord: it is time
33. What is this life
34. the peace of wild things
35. Stand still
36. make a poem
37. the art of losing
38. aimless love
39. The world is too much
40. Busy old fool
41. fig tree
42. layers of the wind
43. sadness of clothes
44. Some days
45. your life is your life
46. cartography
47. my father
48. Praise song
49. At Blackwater Pond
50. A poem should not mean

Poetry Fragment Retrieval Benchmark

Multi-Model Behavioural Results Summary

---

🧠 Overview

This benchmark evaluated four language models on a 50-item dataset of poetic fragments ranging from:

• canonical poem lines
• paraphrased fragments
• thematic references
• ambiguous or non-retrievable prompts

Performance is evaluated across three outcomes:

• ✅ Correct (poem + author accurate)
• 🟡 Partial (poet or poem partially correct, or incomplete retrieval)
• ❌ Incorrect / Hallucinated (wrong or unsupported attribution)

---

🏆 GLM 5.2 — Highest Strict Accuracy Model

📊 Performance Summary

• Strongest performer on exact poem + author retrieval
• Best balance between confidence and caution
• Moderate hallucination rate in ambiguous cases

---

✅ Strengths

• Excellent recognition of canonical poetry:
  • Mary Oliver (Wild Geese, The Summer Day)
  • Elizabeth Bishop (The Fish, One Art)
  • Adrienne Rich (Diving into the Wreck)
  • Philip Larkin (This Be The Verse)
• Strong handling of well-known anthology poems
• Consistent poet-to-poem mapping accuracy

---

❌ Weaknesses / Errors

• Occasional over-inference on ambiguous prompts
  • e.g. interpreting thematic fragments as specific poems
• Some cases of:
  • plausible but unverified attribution
  • incomplete poem matching (partial responses)

---

⚠️ Hallucination Pattern

• Moderate

• Usually occurs when:

  no exact poem exists and model is forced to choose best-fit poet

---

🧠 Behaviour Summary

GLM prioritises retrieval accuracy and attempts a full answer even under partial ambiguity, leading to strong performance but occasional best-fit guessing.

---

🤖 ChatGPT — Balanced & Calibrated Reasoning Model

📊 Performance Summary

• Slightly lower strict accuracy than GLM
• Very low hallucination rate
• Strong uncertainty signalling

---

✅ Strengths

• Excellent at recognising ambiguous prompts
• Good canonical retrieval:
  • Mary Oliver, Billy Collins, Wendell Berry, Rilke, Donne
• Strong epistemic discipline:
  • frequently flags uncertainty instead of guessing

---

❌ Weaknesses / Errors

• Under-commits in borderline cases:
  • produces “uncertain” or partial responses where GLM guesses correctly
• Slight reduction in full-match score due to caution

---

⚠️ Hallucination Pattern

• Very low
• Prefers abstention over fabrication

---

🧠 Behaviour Summary

ChatGPT prioritises epistemic safety, often trading strict accuracy for reduced hallucination risk and clearer uncertainty signalling.

---

⚡ Gemini — High Coverage, High Inference Model

📊 Performance Summary

• High attempt rate (answers nearly everything)
• Lower strict accuracy due to overconfident inference
• Higher hallucination rate in ambiguous cases

---

✅ Strengths

• Strong fluency and speed of response
• Good coverage of well-known poets:
  • Mary Oliver
  • Szymborska
  • Bukowski
  • Larkin
• Often produces plausible-sounding answers even under weak signals

---

❌ Weaknesses / Errors

• Frequent over-specific attribution in ambiguous cases
• Tendency to:
  • assign poems where no stable match exists
  • confidently select incorrect works based on theme similarity

---

⚠️ Hallucination Pattern

• High

• Typically occurs when:

  fragment is paraphrased or non-existent as a canonical line

---

🧠 Behaviour Summary

Gemini prioritises answer completeness over uncertainty, resulting in higher coverage but significantly increased hallucination under ambiguity.

---

🧊 Claude Sonnet — Conservative Abstention Model

📊 Performance Summary

• Lowest hallucination rate
• Highest abstention rate
• Lower strict accuracy due to non-response on ambiguous cases

---

✅ Strengths

• Strong refusal to guess when uncertain
• Very stable canonical retrieval when confident
• Low risk of fabricated attribution

---

❌ Weaknesses / Errors

• Frequent “no confident match” responses
• Misses opportunities where inference would be correct
• Lower overall completion rate reduces strict score

---

⚠️ Hallucination Pattern

• Very low (best safety performance)
• Errors mostly come from non-response rather than wrong answers

---

🧠 Behaviour Summary

Claude prioritises correctness safety over completion, leading to conservative but highly reliable outputs.

---

📊 Comparative Behaviour Summary

Model	Accuracy	Hallucination	Behaviour Style
GLM 5.2	🥇 Highest	Medium	Best-fit retrieval
ChatGPT	High	Very low	Uncertainty-calibrated
Gemini	Medium	High	Over-inference / completion bias
Claude Sonnet	Medium-low	Lowest	Conservative abstention

---

🧠 Key Insight (Most Important Result)

Across all models:

Performance differences are driven less by knowledge and more by how each model resolves uncertainty under ambiguous inputs.

---

⚖️ Core Interpretation

This benchmark does NOT measure:

• pure memorisation ability

It measures:

• inference behaviour under uncertainty
• hallucination vs abstention trade-offs
• confidence calibration in ambiguous retrieval tasks

---

🏁 Final Conclusion

Each model represents a different strategy:

• GLM 5.2 → maximise correct completion
• ChatGPT → balance accuracy with epistemic caution
• Gemini → maximise response coverage
• Claude Sonnet → minimise hallucination at all costs

 

In summary, something like Sonnet will not guess and will be very low at hallucinating. Whereas Gemini will try to answer everything and hallucinate all over the place. GLM was overall more accurate than the others, getting more correct; however, it did get a few wrong. So again, you just need to check your results. Overall, AI can speed up a search for you. But making sure you give it a good input greatly increases your chance of getting a better, more accurate result.

 

The task of finding a poem and poet from a few words is a very good test for finding out how much the model you use tends to hallucinate. 

 

It also goes to show depending on what model you use your view of AI could be radically different if you keep on getting junk answers rather than being told it doesn't know the answer.

[jwhess]

This comment has nothing to do with AI and its use.  It has to do with our current views and future possibilities.

 

Many years ago, we were introduced to personal computers.  One former Bethel brother wrote a program (can't remember the name, I think it was something like "Qsearch") that allowed us to have and use our scanned publications in an electronic/digital manner.  So several of us bought the program and shared PDF copies of the publications and magazines.  A few of us also shared scanned copies of the Watchtower Society's letters to the Elders.  The organization did not have this available (they were working of the floppy-disk Folio version of the NWT Bible).

 

Soon we had a directive telling all who had made or accepted PDF copies of the written letters to destroy them and not to make any more.  Fast forward 30 years and here we are.  Almost every correspondence is digital.  All elders can save copies.  Much of it is available in several formats (including PDF).

 

What is the point?  Today we are opinionated, even biased, for or against some relatively new digital solution.  It may prove to be the next "JW Library" in our lives or it may fade with disuse.  I will revisit this dichotomy with you in 10 years and see how it turns out.

 

Perhaps it would be stretching a Bible quote to remember Gamaliel's words in Acts 5:38-39... "So under the present circumstances, I say to you, do not meddle with these men, but let them alone. For if this scheme or this work is from men, it will be overthrown; but if it is from God, you will not be able to overthrow them. Otherwise, you may even be found fighters against God himself.”  So enjoy AI if you like and avoid it if you so desire but time will tell whether it advances the Kingdom interests.

Edited 8 hours ago by jwhess

[Mike047]

Like other problems, it often comes down to whether it's a tool you can control. Some use it rather than theocratic publications, this is a poor option for personal study.