Copilot Frequently Asked Questions

Learning Conditions that Promote Engagement

What do we mean by learning engagement?

When we talk about engagement, we mean all the actions that students must take in order to learn. Engagement includes paying attention in class, doing homework, studying, asking questions, persisting when work is challenging, and many other behaviors that are critical for learning.

Why is engagement important?

With rare exceptions, students only learn when they engage. This means that academically engaged students are far more likely to learn and succeed in school. Of course, it’s not just about grades or test scores—engaged students are better positioned for a productive, healthy, and intellectually stimulating life. Fortunately, there’s a great deal of research showing what learning conditions cultivate students’ engagement, and, thereby, set them on the path to a more productive, enriching, and secure life.

What are learning conditions?

Simply put, learning conditions are the internal and external experiences students have as they learn. Research shows that many conditions affect students’ desire and ability to engage with learning. When deciding which learning conditions to focus on, PERTS considered several factors:

  • Is there scientific evidence that the learning condition is important for engagement? The condition must have a well-established relationship to student success. You can learn about the research base, by reading the citations listed beneath that condition at perts.net/conditions.
  • Can educators directly influence the learning condition? Many conditions affect whether students engage effectively. Some of those conditions, like hunger and sleep deprivation, can be difficult for educators to directly influence. Other learning conditions, meanwhile, are very much within educators’ sphere of influence. For example, teachers’ decisions affect whether students get the frequent, specific feedback they need to quickly improve their understanding, whether students feel like they belong in class, or whether students believe they can be successful in class.
  • Is it easy to measure? It must be something we can measure regularly enough to see if changes are working. If you’re surprised we’re not measuring something or you think we should consider measuring something in a different way, let us know. As we get more data and more feedback from educators and students, our measures will improve.

Why measure learning conditions?

If we know why students are disengaged, it is much easier to take targeted actions to re-engage them. PERTS helps educators assess students’ perceptions of learning conditions that influence engagement so that educators can zero in on the causes of disengagement and more effectively target their efforts to foster engagement.

How do we measure learning conditions?

We ask students carefully designed questions that have been tested by Stanford University researchers with students and teachers. The questions help educators identify how students’ classroom experiences could be getting in the way of learning. This enables educators to take better targeted actions to influence these experiences and, thereby, increase students’ engagement over time.

These questions enable educators to understand whether or not their student is experiencing a specific learning condition. For example, we ask students to agree or disagree with statements like, “I feel like my teacher is glad that I am in their class.” All of the learning condition questions are listed alongside their learning condition at perts.net/conditions.

We also ask students to explain their answer to one randomly selected question. This provides teachers with more insight into what’s driving students’ answers. And it prompts students to slow down and be thoughtful about their answers.

We designed open ended questions to survey different learning conditions but educators can choose which ones to ask their students.

What else are students asked on the survey?

PERTS surveys periodically ask students several questions in addition to the learning condition questions described above. Some of these questions enable us to disaggregate data in different ways (e.g., to show teachers how students from different groups might experience their class differently). Other questions enable us to continuously improve the quality of the questions we use—or to prepare more comprehensive reports for communities that request them. To keep the surveys short, most of these questions are only asked periodically or only from random subsets of students. These questions inquire about students’ race, gender, prior grades, learning mindsets, comfort answering questions honestly, level of effort in given class, perceptions of their teachers’ mindsets, and other similar factors. You can always preview the survey.

How can I preview the student survey?

You can preview the survey that your team has configured by signing into Copilot, selecting your team, clicking “Settings,” then “Survey Settings,” and then “Preview Survey.”

What if learning conditions improve but students don’t become more engaged?

Fostering student engagement is less like turning on a light switch, and more like growing a garden. It’s a developmental process that takes time and is influenced by multiple factors:

  1. Time Course: Engaged learning is a collection of behaviors that depends on abilities and habits that take time to develop. For example, even if a student decides to study more, it might take her time to realize that she has to start going to the library after school instead of going back to a distracting home environment. Therefore, even if a student is more inclined to be engaged, it might take time for those desires to translate into the habits and study skills that are markers of engaged learning. Give it time.
  2. Multiple Determinants: Creating hospitable conditions for learning is like cultivating a garden: water, sunlight, and a variety of nutrients are each necessary but insufficient for all plants to grow optimally. Successful gardeners do not simply focus on providing sufficient water or providing sufficient sunlight; they attend carefully to each of those conditions with respect to the needs of each of their plants. Furthermore, they are especially mindful of the special care certain plants might need, e.g., because of prior neglect. In the same way, we suggest you work systematically over time to establish multiple hospitable learning conditions. PERTS enables educators to measure multiple learning condition and to try different strategies so that they can take this systematic approach.

Eligibility and Cost

How much does it cost to participate in PERTS programs?

Participation is free for teacher teams (of up to 8 teachers). We also offer additional services to teams who would like more support and training. Teams can ask a professional facilitator to train staff at their school or district. Additionally, teams can set up a multi-team community license (more than 2 teacher teams participating at the same organization). For more information, please contact copilot@perts.net and put the program name in the subject line.

What grade levels can use PERTS programs?

Our surveys are appropriate for students in grades six and up. Reading comprehension of surveys is the main barrier to younger students participating.

What subject areas are the surveys relevant to?

Our surveys are primarily designed for use by academic teachers: those teaching math, ELA, science, social studies, and foreign languages. However, certain learning conditions are highly relevant to non-academic subjects as well. For example, many advisory period teachers rightfully focus on developing strong relationship with their students, and they may find it helpful to assess students’ perceptions of “teacher caring.” Regardless of subject area, we strongly recommend against surveying students in more than 4 of their classes at the same time because of survey fatigue.

Who should be on an engagement team?

Any teacher who supervises one or more classes of appropriately aged students can participate. Teams can have no more than 8 teachers. Typically, a team is made up of 2-8 teachers who teach the same subject or grade-level, but that is not required. In some cases, a teacher will work one-on-one on a team with their instructional coach.

Are PERTS Programs available in multiple languages?

Our materials are only available in English at this time.

Implementation

What is Copilot?

PERTS Copilot is the online platform that hosts PERTS Programs. Teachers use it to collect student surveys, schedule meetings, and generate reports. You can access Copilot at copilot.perts.net.

What is a cycle?

PERTS Copilot programs are organized in cycles of inquiry and action. Cycles typically last 2-6 weeks, depending on the cadence preferred by the team. In order for reports to display properly, teams must enter start and end dates for each cycle.

What if we only want to measure a subset of learning conditions?

In Copilot “Team Settings,” the captain will see a section called “Survey”. In this section, teams can control which learning conditions are measured in their survey. If your team decides not to ask about a particular learning condition, your team can switch it off.

What about survey fatigue?

Students (and adults too) get fatigued by surveys that are long and repetitive. They also grow tired of answering questions if no one acknowledges their responses or addresses their concerns. Unfortunately, many school surveys are designed without regard to students’ or teachers’ experiences, and we (at PERTS) find that troubling.

We would be frustrated if someone asked our opinion over and over five different ways and then gave no indication that anything meaningful would change as a result of our answers. We empathize with the students and with the teachers who are sometimes required to take, or to administer, less-than-useful surveys.

In sharp contrast, we designed our surveys to be part of an empowering process that gives students voice and provides teachers with rapid insights that they can act on immediately. The surveys do two things:

  • They are quick to complete. Surveys take students less than 10 minutes (typically less than 5) to complete.
  • Students’ responses quickly inform teacher practice. We provide guidance to teachers concerning how to frame the surveys to their students (see How do I administer surveys to my students?). When teachers follow this guidance, it ensures their students understand that their opinions are valued by their teachers and that their responses will influence how their teachers conduct class.

How do I administer surveys to my students?

Set up in advance. Make sure each of your students has access to an internet-enabled device like a laptop, tablet, or smartphone. Students do not necessarily need to take the survey at the same time, so it is not necessary for each student to have a device, as long as they can each access a device at some point. Finally, make sure your students have their Roster ID handy so that they can log on to take the survey.

Administering the survey. It’s important to appropriately introduce students to the survey so that they understand and buy into its purpose . You can find detailed recommendations for administering the survey (including unique sign-in codes for each your classes) by following these steps:

  1. Sign into copilot.perts.net and select your team (you’ll be automatically redirected if you’re only on one team).
  2. On the menu to the left, select the cycle for which you are currently surveying students.
  3. Click the “Survey Instructions” button.

If your team has notifications enabled, you will also receive a direct link to your survey instructions in every week during which surveys are scheduled.

Handling makeups. Students who were absent can take the survey at a different time, as long as it is within the same cycle.

Receiving reports: You should receive new reports on the Monday afternoon following your student survey.

What does a PERTS Copilot report look like?

Please see our sample PERTS Copilot report.

Data Disaggregation

Why disaggregate results?

In our school system—and in society more generally—systematic disparities exist between the opportunities afforded to members of different groups. Disparities in opportunity are especially pronounced by race, gender, and socioeconomic status. For example, Black, Latinx, and Native American students are more likely to be suspended than their White peers for the same offenses. They are also less likely to receive the support they need to reach high academic standards. PERTS reports help educators compare the experiences of members of different groups so that, if significant disparities exist, educators can take additional steps to understand and mitigate those disparities.

How does Copilot disaggregate results?

By default, PERTS Copilot reports disaggregate student experience data by gender and by race-ethnicity group membership. Teams can also create a custom target group.

Why are race data grouped in reports?

When students self-identify their race and ethnicity in PERTS surveys, they can choose from 17 different categories and identify with multiple races. However, results in reports are not broken out by those fine-grained categories because that level of disaggregation would make it impossible to maintain the confidentiality of students’ responses. (To maintain students’ privacy, PERTS Copilot reports only show results for a given group when that group has at least five members. That means, for example, that reports could not show results from Native American students if there were fewer than five Native American students in a given class.)

Even though we cannot break out data by each distinct race and ethnicity for privacy reasons, we believe it is important to give educators insight into whether students’ experiences in their classrooms are reinforcing or interrupting the nationally observed disparities in educational opportunity.

As a compromise between the need to maintain student confidentiality and the desire to provide teachers with disaggregated data about opportunity gaps, PERTS Copilot reports group together certain races and ethnicities. Using national statistics on disparities in academic and disciplinary outcomes as a guide, reports group together students who self-identify as Black, Latinx, and/or Native American. Reports contrast their experiences to those of White and Asian students (who are comparatively advantaged according to the same national statistics).

This compromise enables educators to get some insight about opportunity gaps in their classrooms while simultaneously protecting student confidentiality. However, to state the obvious, this approach is limited because the experiences of White students are likely to be different from those of Asian students, and the experiences of Black, Latinx, and/or Native American students are likely to differ as well. Furthermore, opportunity gaps at a particular school might not mirror the national trends. Therefore, we encourage teams to set a custom target group that more accurately represents the groups of students in their own classes who are situated farthest from opportunity.

What is a target group on Copilot?

In many schools and classrooms, specific groups of students are situated further from opportunity. Often, these students are members of specific racial, ethnic or gender groups, English language learners, or students who are members of multiple demographics (for example, boys of color). If teachers are aware that a specific group of students is situated further from opportunity, we encourage them to use the “Target Group” feature in Copilot to define a custom target group that includes those students.

When a target group is set, reports will display results separately for students in the target group and for those who are not. This enables teachers to see whether members of the target group experience their classroom differently. In this way, teachers can recognize whether and how students’ experiences in their own classes may be reinforcing—or mitigating—the opportunity gaps observed in other data. For example, research suggests that teachers who successfully convey a sense of care to African American students are significantly less likely to elicit the defiant behaviors for which African American students are disproportionately disciplined. This implies that teachers can mitigate disproportionate rates of suspensions and referrals by being systematic and intentional about making sure that African American students feel cared for.

The Target Groups feature is intended to support such efforts—to help teachers implement a Targeted Universalist approach towards creating a more equitable classroom climate that fosters engagement in all students.

How do I set up a target group in my reports?

To set up a target group, the team captain should:

  1. Sign into Copilot
  2. Click “Settings” on the menu to the left (you might need to select a team first)
  3. Click “Report Settings”
  4. Define the target group

Privacy

We take privacy extremely seriously for both students and teachers.

What about teacher privacy?

To protect teacher privacy, each classroom entered into Copilot has a main contact who receives survey instructions and reports for that classroom. The only person on a Copilot team who can see a classroom’s reports is the main contact of that classroom. PERTS encourages team members to share and discuss their reports in Cycle Meetings to the extent they feel comfortable doing so.

One side note: it is possible for the team captain and team sponsor to change the setting for any classroom on Copilot, including the main contact. This means that team captain Alice could, in theory, change the main contact of Bob’s classroom to be Alice herself, which would allow Alice to see Bob’s reports. However, Bob will automatically get an email informing him that Alice changed his classroom settings. In other words, there is no way to see another teammate’s class-level reports without having that teammate know that you did so.

What about student privacy?

To protect student privacy and to remove incentives for student dishonesty, Copilot takes several measures to prevent the disclosure of individual students’ responses. With the exception of open response data, we never report data from individual students or subdivisions of students, e.g., by race, if any resulting subgroup would be smaller than 5 students. In some cases, way may also "blur" or "perturb" data (see below) to prevent unintended disclosure. For example, we prevent any metric as showing up as zero percent because that would "give away" all students' responses. Of course, we are careful not to distort data in any way that would meaningfully alter its interpretation.

Blurring is a disclosure limitation method which is used to reduce the precision of the disclosed data to minimize the certainty of individual identification. There are many possible ways to implement blurring, such as by converting continuous data elements into categorical data elements (e.g., creating categories that subsume unique cases), aggregating data across small groups of respondents, and reporting rounded values and ranges instead of exact counts to reduce the certainty of identification. Another approach involves replacing an individual’s actual reported value with the average group value; it may be performed on more than one variable with different groupings for each variable.
Perturbation is a disclosure limitation method which involves making small changes to the data to prevent identification of individuals from unique or rare population groups. Data perturbation is a data masking technique in that it is used to "mask" the original values in a data set to avoid disclosure. Examples of this statistical technique include swapping data among individual cells to introduce uncertainty, so that the data user does not know whether the real data values correspond to certain records, and introducing "noise," or errors (e.g., by randomly misclassifying values of a categorical variable).

— From Privacy Technical Assistance Center (2012). Data De-identification: An Overview of Basic Terms.

You can read our full Privacy Policy at perts.net/privacy.

Can I see individual students’ responses?

We do not permit anyone to see individual students’ responses (with the exception of open ended responses) because doing so could incentivize students to be dishonest and destroy the integrity of the process. Of course, we encourage you to have frank face-to-face conversations with your students, but one of the goals of the surveys is to give you an easy, confidential way to get honest feedback.

Why do my students need to log in to surveys with a Roster ID?

In order to track your students’ data over time and accurately show you how scores change, Copilot needs a way to recognize and distinguish between the students in your class — and across multiple classes on a team. To facilitate this, teachers set up rosters of students for their classes.

What can I use as a Roster ID?

PERTS strongly recommends that you use students’ school email addresses as their Roster IDs. This will make it easier for students to remember their Roster ID, and it will prevent ID conflicts (where two teachers on the same team inadvertently use the same ID for different students, resulting in garbled results).

If you would like to use anything other than a school email address as the Roster ID, please contact copilot@perts.net to confirm the suitability of the alternate Roster ID. As per our strict privacy policy, PERTS only shares student login information with authorized individuals who have a legitimate need to access them, like the students’ teachers (for details, please see our privacy policy).

Communities

Why create a community in Copilot?

Educators using the PERTS Copilot platform typically organize themselves onto teams. However, sometimes an instructional leader or coach may want to group several teams together into a community so that they can monitor and support progress across all of them. For example, say that 24 teachers at the same middle school are all using Copilot. They form three separate teams of 8 teachers — one corresponding to each grade level. If an instructional leader or coach would like to track progress across all three teams, the best way to do that would be to group those teams into a community.

How do you create a community?

  1. Sign into Copilot
  2. Click home
  3. Click add on the “Community” list
  4. Name the community and click save

How do you associate a team with a community?

  1. Get the community code from a community administrator. The community administrator will need to:
    1. Sign into Copilot
    2. Click home
    3. Click into the relevant community from the "Community" list
    4. Click the "add" button
    5. Provide the community code to the captain(s) of the team(s) that will be associated with the community.
  2. Use the community code to associate a team with a community. The team captain will need to:
    1. Sign into Copilot
    2. Click home
    3. Click into the relevant team from the "Team" list
    4. Click the "Settings" on the left
    5. Click "Communities"
    6. Paste in the community code

How do permissions work for communities?

When a team joins a community, it gives the community’s administrator(s) the same permission over its team settings and reports data as are granted to the team’s captain.

Community administrators are able to:

  • See team-level reports
  • Change survey settings and team membership (including who is the team captain)
  • Update class rosters

Community administrators are not able to:

  • See class-level reports (unless they make themselves the teacher for that class, in which case the current teacher will be notified)

Can a team be associated with multiple communities?

Yes, you can associate a team with up to 5 communities. For example, maybe you want your team to be accessible to an instructional coach who works with three teams and also to a district leader who works with twenty teams.

Troubleshooting

How can I help a teacher build student buy-in for surveys?

Survey reports prominently flag problems with student buy-in. For example, they show if students are not comfortable being honest or if they do not believe their teacher will use the data to improve their learning experiences. If a teacher is struggling to get high student buy-in, we recommend their teammates help them by trying some of these strategies:

  • Visit the teacher’s classroom to see how that teacher frames the surveys to students.
  • Work with the teacher and with other team members, as needed, to create a plan for improving student buy-in. (The Survey Instructions page on Copilot has some strategies that have worked for other teachers.)

Why can't I access Copilot?

If you have issues accessing Copilot, your district may have a firewall that is blocking access to the site. Check with your IT department to see if they can resolve the issue. You can test your network access here: perts.net/network-test.

Acknowledgements

Copilot and the learning condition questions and resources were developed by the Project for Education Research that Scales (PERTS), an applied research center at Stanford University. PERTS helps educators apply insights from the learning and developmental sciences so that every learner gets the support and stimulation they need to thrive.

PERTS works with developmental scientists and educators across the nation to build tools and services that help schools foster student engagement and success. Through our work with schools, teachers, and other educational organizations, millions of students around the world have benefited from evidence-based practices.

Key Advisors

Susan Colby, Founder & CEO, Imagine Worldwide

Carol Dweck, Professor, Stanford University

Camille Farrington, Managing Director, UChicago Consortium for School Research

Becky Margiotta, Principal, Billions Institute

Jason Okonofua, Assistant Professor, UC Berkeley

Greg Walton, Associate Professor, Stanford University

Key Funders

The Raikes Foundation

The Overdeck Family Foundation