Skip to Main Content

Evaluating Information: Artificial Intelligence (AI) Chatbots

How to evaluate information

Artificial Intelligence (AI) Chatbots

AI Chatbots, like ChatGPT, are large language models (LLMs) trained to predict likely responses based on patterns in the data they were trained on. 

Potential Uses Current Limitations
  • Help the user generate ideas
  • Provide suggestions
  • Offer insights
  • Chatbots do not possess actual knowledge or understanding
  • Training data may not be up to date
  • Information may reflect biases, misinformation, or subjective viewpoints 
  • Users will need to evaluate the response, and cross-reference information with reliable sources

Speak to your instructor about the use of AI, LLMs, and chatbots in your coursework.

AI for homework? Some DOs and Don'ts by Grammarly 

Prompting AI Chatbots

Clear, concise prompts = relevant AI responses

There are many ways to formulate a prompt. Think of it as a set of instructions for the AI chatbot relative to your information needs. Here are some criteria to consider: 

Effective Prompt = Context + Task + Specificity + Format

  • Context sets the scene and should provide some key information to help clarify the prompt.
  • Task clarifies the action. 
  • Specificity provides adequate context without being overly prescriptive.
  • Format identifies what the output should look like (bullet points, paragraph, etc.), sound like (tone), etc. 

Learn more about prompts and prompt engineering

Master the Perfect ChatGPT Prompt Formula (in just 8 minutes)!

Prompt: Write an outline for a 5 paragraph academic essay that discusses obesity rates among American teens.

ResponseArchived prompt courtesy of ChatGPT

Does it pass the CRAAP test?

Does it pass the CRAAP test?

AI chatbots fail three criteria of the CRAAP test, and are weak in two. 

Currency ChatGPT was trained on data collected prior to September 2021.
Relevance Chatbots cannot always assess context, and prompts may produce LLM hallucinations.
Accuracy Chatbot responses are prone to error.
Authority Language models do not appraise information; chatbot responses may be retrievable, but with limitations. (ChatGPT Shared Links FAQ)
Purpose Chatbots may contain bias and/or subjective information.

Still plan to use a chatbot? APA recommends: 

  • Describing how you used the tool in your Method section or in a comparable section of your paper.
  • Putting the full text of long responses from ChatGPT in an appendix of your paper.
  • For literature reviews or other types of essays or response or reaction papers, describing how you used the tool in your introduction.
    • In your text, provide the prompt you used and then any portion of the relevant text that was generated in response.

How to Cite ChatGPT (April 7, 2023)

APA Style (2023). How to cite ChatGPThttps://apastyle.apa.org/blog/how-to-cite-chatgpt

AI Chatbots in Education

The Future Trends Forum

ChatGPT, part 1 - What might ChatGPT mean for higher education 

ChatGPT, part 2 - What might ChatGPT mean for higher education, continued

ChatGPT, part 3 - Chat GPT and the college curriculum

Harvard Business Publishing 

Unlocking the Power of AI: How Tools Like ChatGPT Can Make Teaching Easier and More Effective

 

Books, Ebooks, & Multimedia

Other AI Tools

Podcasts

Cover for Decoder with Nilay Patel

Verge Editor-in-Chief Nilay Patel talks to a diverse cast of innovators and policy makers at the frontiers of business and technology to reveal how they’re navigating an ever-changing landscape...

Recommended Episodes:

Microsoft CTO Kevin Scott on AI copilots, disagreeing with OpenAI, and Sydney making a comeback

Recode Media: Inside the AI Gold Rush

Exclusive: Google's Sundar Pichai talks Search, AI, and dancing with Microsoft

Hosts Kevin Roose and Casey Newton explore stories from the bleeding edge of tech.

Recommended Episodes: 

Generative A.I is Here. Who Should Control It?

ChatGPT Transforms a Classroom and is 'M3GAN' Real?

Kevin Killed Sydney, and Reddit's C.E.O. Defends Section 230

The Bing Who Loved me, and Elon Rewrites the Algorithm

Personalized GPTs Are Here, F.T.C. Chair Lina Khan on A.I. Competition, and Mayhem at Apefest

Eye On AI

Hosted by Craig S. Smith, Eye on A.I. is an insightful podcast that explores the world of artificial intelligence. With expert analysis and engaging discussions, Craig delves into the latest advancements and ethical implications of A.I., providing listeners with a comprehensive perspective on this rapidly evolving field.

Eye on A.I.

Image result for ezra klein show

The Ezra Klein Show is a thought-provoking podcast hosted by journalist and author Ezra Klein. Through engaging conversations with diverse guests, Ezra explores complex topics, fostering understanding and providing insightful analysis for listeners.

Recommended Episodes:

Best of: Is A.I the problem? Or Are We?

A Skeptical Take on the A.I. Revolution

My View on A.I

Freaked Out? We Really Can Prepare for A.I.

Why A.I. Might Not Take Your Job or Supercharge the Economy

Kara Swisher is a renowned technology journalist and commentator known for her insightful and fearless coverage of the industry.

Open AI CEO Sam Altman on GPT-4 & the A.I Arms Race

Inflection AI

Episode: AI Chatbots and the Humans Who Love Them

Humans are building meaningful relationships with AI chatbots. What will the consequences be?

Your Undivided Attention explore[s] the incredible power that technology has over our lives — and how we can use it to catalyze a humane future.

Recommended Episodes

AI Myths and Misconceptions

Can We Govern AI? With Marietje Schaake

The AI Dilemma

Videos

ECPI's Ethical use statement

Artificial Intelligence.  The growth and development of Artificial Intelligence (AI), such as ChatGPT, presents opportunities and challenges for ECPI University students.  AI provides ways to enhance learning but also has the potential to negatively impact the learning process.  The University recognizes the potential benefits of AI in the learning process but also expects students to independently demonstrate specific skills, competencies, factual knowledge, and general cognitive development within their academic discipline through submission of assignments, exams, and other activities.  Students are accountable for the accuracy, integrity, and originality of work (See Honor Code). The University’s Plagiarism Policy establishes what constitutes the proper and improper use of resources, including AI, and the expectation to properly cite resources used by the student.