Skip to main content

Transparency note: Engagement Assistant

Hannah

General

  1. What is Engagement Assistant? Engagement Assistant provides teachers the ability to auto-generate questions and discussion threads in Interactive Study Material (Document, Video and Audio) based on the context of the material. It is meant to save teachers time as they come up with relevant interactions for students within these assignments. Any generated question also includes a suggested answer, based on the question and the context in which the question appears within the material.
    ​
    Engagement Assistant uses advanced Large Language Models (LLMs) provided by Azure OpenAI Service to process teachers’ input and return the suggestions. More specifically, the text selection and an image of the document’s page where the selection occurs are sent to an LLM to produce the suggested question and answer.
    ​
    We choose to use Azure’s OpenAI Service to deliver this feature because of its enterprise-level security, compliance, and regional availability.
    ​

  2. Is my data sent to OpenAI to train their language models? No. Teachers’ input is only processed by the language models hosted on Azure to generate the suggestions.
    ​The input is never used to train, retrain, or improve any of the Azure OpenAI Service’s Models. It is also never used to train any custom FeedbackFruits AI models. Moreover, we use these models as a “plug-and-play” service. We do not otherwise retrain or fine-tune the models for a specific use case outside of providing custom instructions to the model. In other words, we do not use any user data to retrain, train, or improve our models.
    ​

Transparency, Privacy, and data governance

  1. How are teachers’ data processed and used by Engagement Assistant? Your data is processed in compliance with the the Data Processing Agreement concluded between your institution and FeedbackFruits. Specifically, when using Engagement Assistant to suggest questions, your data is processed in four separate stages (see figure below):

    1. First, your data (the selection you made and portions of the document preceding that point) are sent to the Azure OpenAI Service to be prepared to send into a model; this process is known as tokenization.

    2. Once your data are tokenized and arrive at the model, the model then creates a generation based off custom instructions (created by FeedbackFruits) and an input (your “tokens”).

    3. After the model completes its generation, the instructions, input, and the generation are all sent through the Azure OpenAI Service’s Content Moderation System and Abuse Monitoring System to ensure both the input and the model’s response are appropriate, do not violate any terms of service, and are not harmful to you.

      1. In the Abuse Monitoring System, Azure OpenAI Service may store your input data as well as the model generation for up to 30 days.

    4. Finally, the model’s generation is returned to FeedbackFruits in a standard JSON format from Azure through their API. This only contains the completion data as well as some relevant metadata including reason for stopping and if there was an error.

In addition, all relevant actions performed by or within the Azure OpenAI Service will be done in the region in which you are using FeedbackFruits to comply with data residency requirements and improve latency. Teachers’ input is never used to train, retrain, or improve any of the Azure OpenAI Service’s Models. Currently, we also do not use any teachers’ input to fine-tune the instance of model we use. To read more about how Azure processes and uses your data, please visit Azure OpenAI Service’s page on Data, privacy, and security.

  1. What are the Content Moderation System and Abuse Monitoring System? FeedbackFruits utilizes Azure’s alerting system to ensure that both the input fed into the model as well as the model’s response are appropriate, do not violate any terms of service, and not harmful to you. The alerting system comprises of the Content Moderation System and the Abuse Monitoring System.
    The Content Moderation System is a proprietary collection of models (known as an “ensemble”) that are trained to detect potential abuse, misuse, or harmful content generation within the Azure OpenAI Service. The Azure OpenAI Service does not store any metadata regarding the use of the Content Moderation System. For more information, please visit Azure OpenAI’s content filtering system page.
    The Abuse Monitoring System monitors for any abuse or misuse of the service that could violate Azure’s applicable product terms. Only specific authorized Microsoft employees may review your data in the event that certain content has triggered potential abuse of the system. In the event of misuse of the system, an authorized employee will reach out directly to FeedbackFruits to resolve the issue and prevent further abuse. Azure OpenAI Service will store your input data as well as the model generation for up to 30 days to monitor if there is use of the Azure OpenAI Service that would directly violate Microsoft’s applicable product terms.
    For more information, please visit Azure OpenAI Service’s data privacy page and the Code of Conduct.
    ​

  2. What is included in your prompt? A prompt is a set of specific instructions given to a large language model to generate a particular type of response or follow a specified task. Prompts help prime the AI model to know what and how it should respond.

Examples of prompts include “Write me a short story about turtles and rabbits”, “Explain the moon landing to me as if I were 10 years old”, or “Help me translate my feedback into Dutch.” Because prompts contain cues or guidelines that help create a feature, they may constitute intellectual property owned by FeedbackFruits. Although we cannot divulge the exact prompt, here are some of the elements included in the prompt for the Engagement Assistant:

  • Generate a question and a suggested answer

  • Generate a discussion prompt

  • Ensure that the question can be specified within Bloom’s Taxonomy

  • Include the question and answer in a specific format

  • Ensure the question is answerable within the context (page or text selection) provided

Human agency and oversight

Can I choose to not use Engagement Assistant? When generating a question in Interactive Study Material, you always have the choice to click the “Suggest” button to generate a question. In other words, the question will not generate without an explicit interaction with the button (clicking). When a question and answer are finally suggested, you are also able to change the text, in both, as if they were a draft that you wrote yourself. We do not automatically save the generated question and always give you the opportunity to edit or re-generate to your liking.

Fairness

Does Engagement Assistant work for languages other than English? Not yet. It does read and listen in other languages, but generates content in English. We are working on ways to include other languages within the feature and will update this when we come up with a way that works well within the system while also respecting some of the choices that our users would like to make.

Relevant Azure OpenAI Service Documents

For more information about how data is processed by Azure OpenAI Service, please consult the following resources:

Related articles