Skip to Main Content

Evaluating Information: Artificial Intelligence (AI) Chatbots

How to evaluate information

Artificial Intelligence (AI) Chatbots

AI Chatbots, like ChatGPT, are large language models (LLMs) trained to predict likely responses based on patterns in the data they were trained on. 

Potential Uses Current Limitations
  • Help the user generate ideas
  • Provide suggestions
  • Offer insights
  • Chatbots do not possess actual knowledge or understanding
  • Training data may not be up to date
  • Information may reflect biases, misinformation, or subjective viewpoints 
  • Users will need to evaluate the response, and cross-reference information with reliable sources

Speak to your instructor about the use of AI, LLMs, and chatbots in your coursework.

AI for homework? Some DOs and Don'ts by Grammarly 

Prompting AI Chatbots

Clear, concise prompts = relevant AI responses

There are many ways to formulate a prompt. Think of it as a set of instructions for the AI chatbot relative to your information needs. Here are some criteria to consider: 

Effective Prompt = Context + Task + Specificity + Format

  • Context sets the scene and should provide some key information to help clarify the prompt.
  • Task clarifies the action. 
  • Specificity provides adequate context without being overly prescriptive.
  • Format identifies what the output should look like (bullet points, paragraph, etc.), sound like (tone), etc. 

Learn more about prompts and prompt engineering

Master the Perfect ChatGPT Prompt Formula (in just 8 minutes)!

Prompt: Write an outline for a 5 paragraph academic essay that discusses obesity rates among American teens.

Response: Archived prompt courtesy of ChatGPT

Does it pass the CRAAP test?

Does it pass the CRAAP test?

AI chatbots fail three criteria of the CRAAP test, and are weak in two. 

Currency The data used to train models may be outdated. 
Relevance Chatbots cannot always assess context, and prompts may produce LLM hallucinations.
Accuracy Chatbot responses are prone to error.
Authority Language models do not appraise information; chatbot responses may be retrievable, but with limitations. (ChatGPT Shared Links FAQ)
Purpose Chatbots may contain bias and/or subjective information.

Still plan to use a chatbot? APA recommends: 

  • Describing how you used the tool in your Method section or in a comparable section of your paper.
  • Putting the full text of long responses from ChatGPT in an appendix of your paper.
  • For literature reviews or other types of essays or response or reaction papers, describing how you used the tool in your introduction.
    • In your text, provide the prompt you used and then any portion of the relevant text that was generated in response.

How to Cite ChatGPT (April 7, 2023)

APA Style (2023). How to cite ChatGPT. https://apastyle.apa.org/blog/how-to-cite-chatgpt

APA Style Guide

AI Chatbots in Education

The Future Trends Forum

ChatGPT, part 1 - What might ChatGPT mean for higher education 

ChatGPT, part 2 - What might ChatGPT mean for higher education, continued

ChatGPT, part 3 - Chat GPT and the college curriculum

Harvard Business Publishing 

Unlocking the Power of AI: How Tools Like ChatGPT Can Make Teaching Easier and More Effective

 

AI LibGuides - Artificial Intelligence Now: ChatGPT & AI Literacy Toolbox - FIU Libraries at Florida International University

Code.org

Discover AI for Educators

Georgetown AI (Generative) Resources LibGuide

University of Waterloo ChatGPT and Generative Artificial Intelligence (AI) LibGuide

Carnegie Melon AI Literacy Resource Hackathon

Books, Ebooks, & Multimedia

Other AI Tools

Videos

ECPI's Ethical use statement

Artificial Intelligence.  The growth and development of Artificial Intelligence (AI), such as ChatGPT, presents opportunities and challenges for ECPI University students.  AI provides ways to enhance learning but also has the potential to negatively impact the learning process.  The University recognizes the potential benefits of AI in the learning process but also expects students to independently demonstrate specific skills, competencies, factual knowledge, and general cognitive development within their academic discipline through submission of assignments, exams, and other activities.  Students are accountable for the accuracy, integrity, and originality of work (See Honor Code). The University’s Plagiarism Policy establishes what constitutes the proper and improper use of resources, including AI, and the expectation to properly cite resources used by the student.