Craft a productive topic of engagement and responsive assessment with contextual knowledge and broad context (2023)

The Research and Theory Directorate (RTD) supports the core mission of the Association for Educational Communication and Technology (AECT). Department:

• It facilitates the development and advancement of theory.
• It promotes, presents and disseminates research and science that encompasses multiple perspectives.

It supports the study of social and cultural issues in this field.

• It supports, develops and mentoring young scientists (AECT, 2023).

These goals are in line with the ongoing development of the broader association. RestructuredHandbook of Educational Communication and Technology ResearchIn the fifth edition (Bishop et al., 2020), the shift from education toTechnologyand educationAsk.withinpreludefourth edition, Reeves and McKenney note that "much research in educational technology and communications has focused on 'things' rather than 'problems'. problems and their solutionsinnovatorplan andsuitableTechnology” (p. VI, emphasis mine) Reeves and McKenney point to longstanding issues such as racial justice, teacher education, sustainability and external partnerships as areas in which AECT members should work.

In the spirit of innovative design and appropriate technology, this paper summarizes an integrated approach to teaching and assessment that my colleagues and I have citedparticipatory learning and assessment,ThePeople's Liberation Army.This approach stems from two decades of design-based research. As will be discussed, the first decade involved research-practice collaborations with leading innovators in the fields of educational multimedia and educational video games. The second decade of research was dominated by online learning using widely used tools.

This research focuses on enduring questions that transcend technology and are arguably beyond the reach of previous theories. Some of these problems are relatively common:

• Balance between intrinsic and extrinsic motivation.
• Balancing formative and summative scoring functions.
• Raise grades without taking the teaching test.
• Overcoming historical and persistent discrimination and inequality.

Other questions addressed in this study relate specifically to online learning.

• Minimize teacher burnout from excessive teacher-student interaction.
• Reduce isolation and increase social engagement for online learners.
• Supports online peer-to-peer interactions without synchronous interactions.
• Ensure valid online appraisal credentials without costly online appraisers.

This research draws heavily on contemporary cognitive constitutive theory (ie Lave & Wenger, 1991; Greeno, 1998). The work goes beyond the broad characteristics of the situationlearnIn relation to “real” and “real” situations in teaching and assessment (e.g. Herrington et al., 2014). In particular, this work builds on design principles that emerged from the research of Randi Engle (1965-2012) and colleagues. These principles advise caution when relying on real-world frameworks defined by subject matter experts. Instead, these principles advise learners to problematize (i.e., contextualize) the lesson from their own perspective (which experts may or may not believe to be true).

In two "Expert Consensus Research Reports", the study addresses a fundamental issue of the National Research Council (NRC) and its successor organization, the National Academies of Sciences, Engineering and Mathematics (NASEM). These reports were prepared by a carefully selected panel of experts following a rigorous process that reached consensus within the panel at the time.How people learn: brains, minds, experiences and education(NRC, 2001) provided a consistent view of learning and transfer that strongly reflected the social-constructivist orientation of the committee's co-chair John Bransford and was embraced by many cognitive scientists and educational psychologists of the time. its title and contentHow People Learn II: Learners, Situations and Cultures(NASEM, 2018) reflects a “social turn” that was already underway when “HPL I” was designed (e.g. Gee, 1999). HPL II reveals a new consensus that (a) context and culture matter in learning and (b) context and culture matter far more than a similar panel of experts that closed two decades ago. It is worth noting that there is a chapter on learning transfer in HPL I, butNOa chapter on culture, while HPL II contains a chapter on culture, butNOA chapter on transportation. This suggests a lack of consensus, and perhaps even open disagreement, about the implications of this social shift for our theory of empathy. This is important because our transfer theory has profound implications for the design of instruction and the assessment of learning and testing performance. The research summarized here attempts to provide insight into the implications of transfer case theory for teaching, assessment, and testing in the light of other, more established theories, thereby supporting a new consensus (Hickey & Pellegrino, 2005).

Responsive "multi-level" evaluation

The first phase of this research "matched" formative and summative features at multiple "levels" of increasingly formal assessment. This study builds on the assessment levels used in the more naturalistic assessment of the STEM curriculum by Ruiz-Primo et al. (2002) and the innovative "dialogue" approach to formative assessment in Duschl & Gitomer (1997). This phase involves design-based research in collaboration with faculty and students learning to leverage innovations from leading technology innovators. The fundamental approach that emerged from his researchGenetic OscilloscopeA multimedia introductory genetics course developed by Paul Horwitz (Hickey, Kindfield, et al., 2003). The method was quasi-experimentally validated in the GenScope study (Hickey & Zuiker, 2012),Find AtlantisVideo games (Barab et al., 2007; Hickey, Ingram-Goble & Jameson, 2009) and NASA-funded multimedia STEM projectsfuture class(e.g. Hickey, Taasobshirazi and Cross, 2012). This approach has been further refined in the participatory research on new digital media (Hickey, McWilliams, & Honeyford, 2011; Hickey, Honeyford, & McWilliams, 2013) and VET assessment (Hickey, 2005) developed by media scholar Henry Jenkins.

Central to this approach is the idea that a broader view of learning allows assessment itself to serve as a summaryModelA formative function of one form of learning and a formative function of another form of learning. This circumvents the social constructivist focus of this summative assessmentPurposeDestruction of formation purposes (e.g. NRC, 2001; Pellegrino, 2002). In most studies, the four levels of assessment include:

• DirectInformal language assessment.
• keep your mouth shutWorry about the bond.
• proximalUnderstand the rating.
• leaveSuccessfully tested.

For example, narrow reflection on engagement may summatively assess past engagement, while conceptual understanding is formatively assessed. Changing the representation of knowledge at each level requires students to transfer the knowledge they gained from formative feedback at the previous level. refer directly to Messicks (1994), who is frequently citedEase of constructing irrelevance(i.e. “teaching to the test”), which maintains the validity of grades as essential evidence of learning. Because students are not directly prepared for each exam, these scores are a reliable estimate of transfer in assignment grading, grade measurement, and course evaluation.

Productive engagement and participatory learning and assessment

Our approach was originally called PLA and was further theorized around 2009 when we transitioned to online learning. This approach is considered “responsive” because it is responsive to the different forms of knowledge being assessed at each level. As Table 1 shows, assessments at all levels increasingly capture (a).formalrepresentation of knowledge, (b) broader positioning of the school, (c) increasing privacyinteractive(after Hall & Rubin, 2013) and (d) longerTime scheduleLearning (according to Lemke, 2000).

Table 1

Responsive Assessment – ​​Online Learning

Following James Gee's suggestion, we have design principles inProductive disciplinary engagement(PDE, Engle & Conant, 2002) in these multi-level assessments. These principles suggest that learners (a)they are disturbingFrom their point of view, (b) to giveStrengthsolve the problem that arises, (c) is maintainedresponsibleIn disciplinary matters and (d) are givenResourceNecessary for the implementation of the first three principles. This synergy leads to a comprehensive PLA framework as introduced by Hickey and Rehak (2013). This new framework includes the following five legal design principles:

1. Use a common framework to give meaning to knowledge tools.
2. Recognize and reward productive discipline participation.
3. Artifacts are valued with local reflection on engagement.
4. Let people judge their understanding privately.
5. Measure performance skillfully when necessary and appropriate.

In short, the first two PLA principles comprise Engel's PDE principles and the first level of assessment, while the last three PLA principles represent the other three levels of assessment.

A central idea of ​​PLA is that a shared (classroom) environment of student engagement is possible and meaningful when each student works on content from their own perspective. Once each student has identified an initial framework for problematizing course content, peers can learn from each other using examples and ideas, but must place them in their own context.

This public engagement was originally supported by an open source wiki toolSakaiLMS, we call them “wikifolios”. Then we end up relying on Google Docs (“G-Portfolios”) or the headlines of the new Canvas and Blackboard LMS discussion boards (“E-Portfolios”). Use Google Docs and forum titles to support discussions through thread comments from teachers and students. But traditional forums were never used for meaningful discussion in the PLA. This reflects concerns expressed by Thomas (2002) and others about the incoherence and abstraction of traditional forums. Instead, most interactions take place through direct comments on student work or through social annotations on learning resources. In many cases, regular linked instructor announcements support additional targeted interactions. This particular interaction with field artifacts and student comments supports an important finding that emerged early in this work:Most students enjoy participating in discussions where their own work and ideas come into play.For this reason, student contributions or discussions are never graded and are rarely required. This minimizes the sad conversation that mandatory and rated posts often cause.

Extend PLA via the extension framework

design principlesexpansion frameEmerges when Engle (2006) reanalyzes student engagement and assessment performance from the original 2002 study. Engle looks for interactions that are responsible for "generating" learning and that can be easily and comprehensively translated into an assessment of performance that is very different from the learning environment. He concluded that when students are (a) pushed to find diverse connections to people, places, subjects, and times outside the scope of the course and (b) encouraged to position themselves as responsible writers, greater Societies contribute to dialogue across time and space. These principles were supported in the experimental study by Engle et al. (2011) compared a broad-based biology class to a “bounded framework” in which students are not asked to make connections or identify themselves as authors.

Engel et al. (2012) further elaborated the extension framework. They explain how it supports the extension frameworkin progress(Bloom et al., 2009). Here the learners make so many connections between their past experiences, current external goals and the future context that the learning process becomes part of a larger “inclusive context”. Experienced writers see this situation as the epitome of writing. To assist students in this seemingly elusive goal, Engle et al. (2012) provide five compelling explanations for why extended frameworks and sequentialization should support genetic learning:

1. More interaction between settings.Helping students visualize how they will use what they have learned in the future will naturally lead them to adopt more productive learning strategies that go beyond completing tasks.
2. recognize relevance.When learners are able to visualize themselves applying what they have learned, they are more likely to recognize these transfer opportunities when they present themselves.
3. transfer to prior knowledge.A third interpretation of the expanded framework is to position students as authors of their own ideas, which would encourage them to bring in more relevant past experiences. If peers find these experiences useful for their own learning, learners are encouraged to transfer more of their past experiences.
4. Take responsibility for the content.When students have successfully written new content at the intersection of their experiences and course content, they are more likely to feel “ownership” of that content. This can inspire confidence when using this knowledge in potential transmission environments.
5. Summary of the letter.If students take sufficient opportunities to create new knowledge, this experience should be generalized to other knowledge in the field and eventually to knowledge in new fields.

Engle and colleagues provide empirical examples from other studies to support each of these explanations. Apparently, in Engle's final article before her untimely death, the 2012 paper also included a possible research agenda for extended frames in the form of three types of studies that could be used to study how extended frames affect transmission. These studies includedisentanglement experiments, comparative classroom studies,AndMicrogenetic Research.Essential points of our research program aim to lay the theoretical and methodological foundations for the continuation of this work and to transfer this research to an online setting.

Incentive Effects of PDE, Extended Framework and PLA

Before considering our present and future efforts, we note that the work embedded in and (sometimes) related to this assessment work is an occasional review of motivation. This thread grew out of previous efforts to reshape the study of motivation from examining individual factors to broader social and cultural factors (eg, Hickey, 1997; Hickey & McCaslin, 2001; Sivan, 1986). This work aims to help resolve a heated debate about the influence of "extrinsic" rewards on "intrinsic" motivation (eg, Cameron & Pierce, 1994 vs. Ryan & Deci, 1996). The reader is reminded that hundreds of experimental studies on the so-called "overrationalization effect" (Lepper et al., 1973) have shown that when students receive arbitrary "extrinsic" rewards for their actions, their intrinsic motivation and engagement decrease. effort (Tang & Hall, 1995). The reader is also reminded that this seemingly intractable debate demonstrates the inability of experimental and meta-analytical approaches to answer these fundamental questions rooted in competitive learning theory (see Lepper et al., 1999).

One of the most important but underestimated aspects of context theory is at the center of motivational work and evaluation. so calledfree compositionIt's about reconciling the tension between (a) how people behave and process information, (b) how people understand the world around them, and (c) how which social and cultural systems function. As introduced by Greeno and Moore (1993) and elaborated by Greeno (1998), the popular "aggregative" reconciliation uses the aggregated activities of individuals to explain the functioning of larger sociocultural systems. In terms of motivation, this is perhaps best illustrated by Bandura's (2000) theory.collective effectiveness,Among them, group efficacy consists solely of the self-efficacy of the group members. This concentrated description of social and cultural activities assumes the absence of a "disembodied group spirit" (p. 76) independent of the collective beliefs of the members.

Instead, legal constitutions describe individual activities as "special cases" of sociocultural activities. In other words, human behavior and information processing,AndUltimately, the formation of human meaning is best explained through participation in social and cultural practices. An important caveat here highlights the difference between the work of learning scientists and that of cognitive scientists. As a student scientist, I am always looking for new solutions to traditional problemsassumeThe legal component is true (rather than claiming that it isAndreal). In doing so, I avoid more fundamentally naturalistic debates about the nature of human cognition, such as the much-quoted debate between Anderson et al. (1996) and Greenow (1997). Instead I asked ifassumethis inherentAndExtrinsic sources of motivation are all special cases of social activity, and they provide a new way forward in this seemingly intractable debate (Hickey, 2003). In practice, this means that we need to go beyond the usual research on sociocultural influences on individual motivation and examine how sociocultural and situational learning theories can lead to entirely new theories of motivation (summarized in Hickey & Granade, 2004) and class arrangements for such subjects. Hickey and Shaffer, 2006)

This new way of thinking about extrinsic motivation was explored in a quasi-experimental study on two carefully matched class pairs at Atlantis. Students in both classes were provided with external incentives (i.e. "badges" for their physical and virtual avatars). In contrast, students in the other two grades were encouraged to satisfy their curiosity and interests (adapted from Lepper & Malone, 1987). Multiple levels of assessment and incentives showed that students in the extrinsically motivated condition made significantly greater progress in subject learning and showed significantly increased engagement and interest in the subject. Rejecting the overrationalization hypothesis, students receiving extended rewards showed slightly larger gains Personal interest in the subject (Hickey & Filsecker, 2012; Filsecker & Hickey, 2014)

In a subsequent research program, random synthesis revealed promising new ways to use and study digital "tokens" and other "micro-evidence". Unlike traditional assessments and certificates, these new open digital badges can include actual competency claims and links to web-enabled digital credentials that support those claims. As discussed in Casilli and Hickey (2016), these developments have far-reaching implications, as they undermine the value of traditional methods of verification (evidence) and authentication (credibility value) while increasing the perceived importance ofreliability(This has long been dismissed as an element of "face" validity, Popham, 1990). Of course, these new methods of identifying learning and achievement have helped reignite the debate about extrinsic motivation. After extensive coverage of the national competition, several influential observers argued that such signals were external in nature (e.g. Jenkins, 2012; Resnick, 2012). But a longitudinal study of thirty signaling systems funded by a national competition (Hickey, Willis & Quick, 2015) identified four distinct signaling systems:

• ability basedA system of awarding badges for personalized knowledge of specific skills. These signals are linked to external (i.e., “external”) incentives, such as B. Course credits or new opportunities.
• after completionThe system awards badges for the completion of complex studies, projects and portfolios. These signals tend to focus on intrinsic sources of motivation, such as interest and curiosity.
• attendance basedA system for rewarding badges for participation in social learning and group projects. These signals are arguably more focused on 'committed engagement', a more sociocultural form of motivation introduced by Hickey (2003).
• hybrid The system awards badges for two or three of the above functions.

A follow-up study found that equity-based token schemes are more likely to lead to thriving token-based “ecosystems” after initial funding is exhausted. Most other badge systems have proved unable to sustain (or in some cases even implement) their ambitious individualized assessment programs for competency or degree (Hickey & Chartrand, 2020).

This work reflects the value of new contextual synergies involving motivation and evaluation and has a wide range of design principles for "where signaling works best" (Hickey & Willis, 2015), a new functional framework for signaling (Hickey, Uttamchandani, & Chartrand, 2020) and seven “new arguments” for motivating learning with digital signage. As detailed in Hickey & Schenke (2019), these new arguments are:

1. Digital signals have an intrinsic meaning.Regarding traditional incentives and certificates,DigitalTrademarks may contain specific claims and evidence to support those claims, making them more meaningful.
2. Open digital signals are particularly important.Compatible with the Open Badges Infrastructure metadata system, digital badges can be easily shared and read by humans and machines, and circulated on social networks for added meaning.
3. Open digital tokens are very important credentials.The newly introduced "Signature" feature allows open themes for digital badges, invites organizations to officially endorse a class of badges before issuance, and then authorizes individuals (such as members or the organization) to verify that they are in the provided evidence is included.
4. The negative consequences of extrinsic rewards are overestimated.While the over-justification effect is real, the focus on it ignores other relevant factors. These include the potentially positive short-term behavioral consequences of extrinsic motivation, the seemingly positive neurological consequences (Hidi, 2016), and their combined potential to increase “community” motivation. These other factors seem likely and even probable to outweigh any possible negative consequences arising from the overjustification effect.
5. Focus primarily on social activities, followed by personal activities.This comes straight from the casual composition above. Theoretically, this means understanding both extrinsic and intrinsic motivation as special cases of social engagement. In practice, this means examining the effects of signaling systems on student discourse before considering their effects on individual behavior or knowledge.
6. The entity participation model is ideal for studying digital credentials.Most open digital signage systems are used to mark learning on digital networks, most of which have a social component. Brown and Adler (2007) argue that digital social networks represent a new form of engagement that goes beyond traditional models that focus on individual behavior or knowledge.
7. Explore incentives and digital certificates at three increasingly formal levels.According to research reported by Filsecker and Hickey (2014), this means examining the effects of cues on (a) productive disciplinary engagement, (b) intrinsically motivated engagement and interest in the situation, and (c) personal interests and broader education of change in ecosystems.

recent and current efforts

Efforts have been made in recent years to use PLA to design a number of online learning frameworks. These include the study of undergraduate theoretical courses (Chartrand et al., 2020), postgraduate courses in pedagogy (Hickey, 2015a; 2015b), fully online secondary schools (Itow, 2018), self-paced technical training courses (Hickey, Duncan, et al. People , 2020) and “mega” online open courses completed by hundreds (rather than thousands) of open learners (Hickey & Uttamchandani, 2017).

Another new attempt by us is to introduce the PLA framework to a wider audience in a less theoretical way. Hickey, Chartrand, and Andrews (2020) translate five PLA design principles into 15 discrete steps for use by instructional designers that are clearly more based on cognitive-constructivist learning theory. Hickey, Duncan and others. (2020) introduced a subset of these steps for G-Portfolio course design. Presented without any design principles or theory, these steps are intended for educators who are going online but have little or no background in learning theory. These two articles are helpful for two self-paced professional development courses used by hundreds of middle school teachers who have transitioned to online learning during the pandemic (Chartrand & Hickey, 2020; Hickey & Harris, 2020).

One of our current tasks is to promote new design principles for specific target groups. Hickey and Harris (2021) extend PLA to include a new set of design principles specific to online course assessment:

1. Embrace occasional reconciliations rather than cumulative ones.
2. Focus on evaluating the function and not the purpose.
3. Engage in multiple complementary ways of interacting with students.
4. Increasingly use formal assessments to capture longer periods of learning.
5. Embrace transformative functionality and system effectiveness.
6. Empower students to be responsible writers.
7. Relocate minority students for equitable participation.
8. Make evidence more meaningful to designers, reviewers, and researchers.
9. Improve the reliability of results and the effectiveness of teachers.
10. Increase the credibility of student ratings and grades.

Readers may notice that this collection combines principles from our previous efforts and new principles from other current efforts (described below). This synergy illustrates an important point of design-based research - that it is constantly evolving.

This new work encompasses contemporary 'wealth-based' responses to historical and persistent inequalities and discrimination, offering practical and theoretical alternatives to outdated 'deficit-based' responses. This work draws directly on the critique of Engle's PDE principles by Agarwal and Sengupta-Irving (2019). Referring to the four PDE principles summarized above, they first recognize that (1) problem solving invites students into their unique cultural and linguistic experiences, (2) providing institutions to explore these problems allows students to taking an active role in building knowledge, (3) accountability helps ensure that power goes hand in hand with valid reasons for criticism, and (4) the provision of culturally relevant resources responds to the interests of diverse students.

The admission of Agarwal and Sengupta-Irving (2019) is important because they argue that PDEs are inherently more asset-based than most contemporary (e.g., social-constructivist) educational frameworks. They emphasize advances in the science of learning in this regard and question the extent to which existing PDE principles will support students who identify with minorities. Teachers and peers who are more favored and/or privileged see them as lazy or disruptive (eg, Paris, 2012). This means that the application of PDE (and by extension extended context and PLA) while ignoring the realities of power and privilege limits support for justice. For example, Agarwal and Sengupta-Irving argue that problematizing content in ways that challenge culturally dominated forms of knowledge can lead to racial tensions that are detrimental to minority students. In response, they introduced new onesConnectivity and productive disciplinary engagementMinority students are invited to identify as Principles (CPDE) with unique experiences and (hence) expertise. These principles are (1) to use sociopolitical uncertainty (SPU) to problematize disciplinary knowledge, (2) to limit inappropriate social power, (3) to ensure fair accountability, and (4) to resolve sociopolitical controversies as a resource.

We are refining a postgraduate course in educational assessment using CPDE principles that has served as a test bed for PLA ideas and practices. During three annual iterative improvement cycles, we tested new course features and then incorporated effective features into all assignments (Hickey & Lam, 2022, 2023). This work resulted in a new framework that we callCultural Sustainability Education Assessment(CSEA) includes a new set of assessment design principles:

1. Includes optional Socio-Political Uncertainty (SPU) that invites, but does not require, students who identify with minority groups to position themselves as having valuable relevant experience and expertise.
2. The teacher's informal assessment of reasons for disciplinary action should be included in theatThe students gain valuable experience and specialist knowledge and, if necessary, are carefully introduced to other SPUs.
3. Students are encouraged to think about how social and cultural factors affect their participation without asking them to talk about their communities.
4. Teachers should read Cultural Reflections and respond immediately to particularly compelling content in public or private comments, whichever is appropriate.
5. Include a formative self-assessment for all students who choose conflict to effectively build familiarity with these issues.
6. Add exam questions that score some (but not all) of the confrontation and the SPU and make sure they work properly.

Preliminary evidence from student engagement and anonymous course reviews suggest that this approach was indeed successful. For example, these new features seem to encourage students to introduce new SPUs to their assignments and uncover minority identities (like LGBTQ+) that would otherwise remain hidden.

Another part of our current job is making sure Randi Engle gets credit for the ideas we love to come up with. While Engle (2011) summarizes applications of PDE principles (and other related research) as of 2010, much of the work on PDEs and extended frameworks has not been systematically reviewed. To this end, Hickey, Chartrand, and Harris (2021) systematically analyzed 31 empirical studies that contain these terms in their titles or abstracts. We are currently conducting a thematic review of 2715 peer-reviewed publications covering both conditions. As part of this work, Freedman et al. (2023) analyzed 32 publications on diversity, equity and inclusion thematically, while Harris et al. (2023) theme analyzed some of this research on teacher education.

In the spirit of supporting Engle's legacy, we recently published a substantial entry in the Educational Encyclopedia recognizing Engle's contributions (Hickey, 2022). Finally the next chapterHandbook of Educational Psychology(Hickey & Lam, in press) advocate a new transport model that consistently incorporates ideas from Agarwal and Sengupta-Irving. This brings fairness and justice to the decentralization debate for the first time. In this way we hope that both lines of thought will find their way into a new legal/sociocultural consensus on transfer theory.

Estimates, concessions, puzzles and conclusions

My colleagues and I are deeply grateful that these efforts were recognized by the 2022 runners-upTheory Spotlight AwardFrom the AECT Department of Research and Theory, we also thank the institutions, institutions and university initiatives (listed above) that have supported many of these efforts, as well as the students, other departments and administrators at the schools and universities for their support and (sometimes) Patience in conducting this research.

We recognize that some of our ideas conform to (and potentially challenge assumptions about learning theory and practice) held by many AECT members and the broader learning design and technology community. We anticipate that there will never be a "sociocultural revolution" to overthrow or replace prevailing theory and related practices. For complicated reasons, this actually agrees with our goals and the arbitrary combinations summarized above.

While being marginal here and here, we believe that context synthesis can help solve another theoretical puzzle in the LDT community and beyond. This conundrum involves the divide between proponents of social-constructivist theory and practice and those of cognitive association theory in favor of teachers using artificial intelligence, personalized adaptive learning technologies, and competency-based education. This split helped fuel the dissolution/renaming of the International Association for K-12 Online Learning (iNACOL) in 2018 and the formation of several new organizations. With continued support from the Gates and Zuckerberg foundations, proponents of iNACOL's competency-based personalized learning founded the new Aurora Institute and expanded their focus beyond technology-based learning (Barnum. 2018, 2019). iNACOL members (e.g. Archambault et al., 2022) who support socially constructive frameworks for studying communities join the newly formed onesNational Quality StandardAnd helped draft a new set of standards for online instruction (NSQ, 2018) that emphasize collaborative learning and teacher-student relationships over individualized competency-based learning.

On the one hand, skeptics question the theory and evidence behind competency-based personalized learning (e.g. Enyedy, 2014; see Barnum, 2020; Pane, 2018; Pane et al., 2017). Proponents, on the other hand, cite evidence (e.g. Koedinger et al., 2010; Connections Academy, 2018), while powerful institutions and many policymakers appear to be convinced of the potential of personalized learning. In fact, our own research (Hickey, Robinson, et al., 2020) found that intelligent tutoring systems were significantly more effective and efficient in helping students successfully complete introductory STEM courses at undergraduate level compared to supportive "developmental" education. More realistically, thousands of the K-12 students in the US already attend full-time virtual schools or take virtual classes from K-12 Stride Inc. or Connections Academy Inc. that rely heavily on personalized computer-based adaptive learning technologies. It seems that our academic programs could encourage and study these methods more systematically.

In conclusion, we salute and encourage the many efforts being made by the LDT and related communities to address the discrimination and injustice that continue to plague our education system. Although much remains to be done, in recent years there have been significant advances in LDT (e.g. Glazewski and Ertmer, 2020), learning science (e.g. Esmonde and Booker, 2016) and educational psychology (e.g. Zusho and Kumar, 2018). ). A key question worthy of further theoretical and empirical examination is whether the teacher's "agency" narrative fostered by competency-based personalized learning is indeed the most effective and appropriate response to these challenges (as suggested by Sturgis & Casey, 2018, see Aurora Institutes , 2023). No class of approaches seems to address this complex and multifaceted problem. All aspects require sustained, coordinated and well-supported efforts.

bibliography

Agarwal, P. and Sengupta-Irving, T. (2019). Joining forces to advance research towards cohesive and productive disciplinary engagement in mathematics and science.knowledge and teachings,37(3), 349-366.

Anderson, J.R., Reder, L.M. and Simon, H.A. (1996). internship and training.educational researcher,25(4), 5-11。

Archambault, L., Leary, H., and Rice, K. (2022). Pillars of online pedagogy: A framework for teaching in an online learning environment.educational psychologist,57(3), 178-191.

Educational Communications and Technology Association (2023, March 1).Mission Statement of the Department of Research and Theory[Website].https://aect.org/rtd.php

Aurora Institute (2023, March 1).The importance of diversity, equity and inclusion in education[Collection of sites and resources].Author.https://edtechbooks.org/-sPPB

A. Bandura (2000). Exercising human agency through collective effectiveness.Current directions in psychology,9(3), 75-78。

Barab S, Sadler T, Heiselt C, Hickey D and Zuiker S (2007). Relevant Narratives, Studies, and Inscriptions: A Framework for Social Science Research.Journal of Science Education and Technology, 16,59-82。

Barnum, M. and Darville, S. (2018, September 10). Discover the latest big donor to education: The inside story of Chan Zuckerberg's $300 million school rehabilitation campaign.Chalk.https://edtechbooks.org/-LzLm

Barnum, M. (2019, October 28). Online learning group iNACOL is being revamped and says it's less tech-centric. More about transformation.Chalk.https://edtechbooks.org/-YaLa

M Barnum (2020, May 28). Pandemic promotes "competence-based" learning, thinking elements remain rare.Chalk.https://edtechbooks.org/-wWsoE

Bishop , M. J. , Bolling , E. , Ellen , J. and Islands , V. (Eπιμ.) (2020).Handbook of Educational Communication and Technology Research.(fifth edition). jump for joy.

Bloome, D., Beierle, M., Grigorenko, M., and Goldman, S. (2009). Learning Over Time: Using Continuing Collective Memory and Lesson Plan to Create Learning Opportunities in the Ninth Grade Language Arts Classroom.language and education,23(4), 313-334。

Brown, J., & Adler, R. P. (2008). Open Education, Long Tail and Learning 2.0.Educational career review,43(1), 16-20。

Cameron, J. & Pierce, W.D. (1994). Reinforcement, rewards, and intrinsic motivation: A meta-analysis.Review of educational research,64(3), 363-423。

Casilli, C and Hickey, DT (2016). Go beyond traditional certification and assessment paradigms with informative digital badges.information society, 32(2), 117-129。

Chartrand, GT, Andrews, CD, Hickey, D.T. (2020). Shaping Efficient Online Learning: A Legislative Research Agenda. In B. Hokason et al. (editor-in-chief),Interdisciplinary: Educational Communication and Technology: Problems and Innovations(pp. 81-92). jump for joy.

GT Chartrand und DT Hickey (2020)Responsive Engagement and Virtual Student Assessment (REVLA).IU Expand's open online self-study course for educators on canvas.https://bit.ly/CanvasREVLA

Liaison Academy (2018).New research on the effectiveness of Connection Academy's virtual public schools shows positive academic results.[press release].https://edtechbooks.org/-GIxQF

Duschl, R.A., & Gitomer, D.H. (1997). Strategies and challenges to change the focus of assessment and teaching in science education.educational assessment,4(1), 37-73。

Engle, R. A. (2006). Structuring Interactions for Efficient Learning: A contextual account of transfer in a classroom community of learners.Journal of Learning Science,15(4), 451-498.

Engle, R. A. (2011) A productive framework for disciplinary engagement: origins, key concepts, and development. In Y. Dai (Editor-in-Chief)Design Research for Learning and Thinking in Educational Environments: Fostering Intellectual Development and Function(S. 161-200).Routledge.

Engle, R.A. & Conant, F.R. (2002). Guiding Principles for Fostering Productive Disciplinary Engagement: Explaining debates that emerge in classroom learning communities.knowledge and teachings,20(4), 399-483。

Engle, R.A., Lam, D.P., Meyer, X.S., and Nix, S.E. (2012). How does expanding context make migration easier? Several proposed explanations and a research agenda to explore them.educational psychologist,47(3), 215-231。

Engle, R.A., Nguyen, P.D. and Mendelson, A. (2011). The effect of context on transfer: preliminary evidence from a classroom experiment.teach science,39, 603-628。

Enedy, N. (2014)。Personalization of Instruction New interests, outdated rhetoric, limited results and the need for new directions in computer-based learning.National Center for Education Policy.https://edtechbooks.org/-IWwG

Esmonde, I. & Booker, A. N. (Hrsg.) (2016) The power and privilege of the science they have learned.Taylor und Frances

Filsecker M, Hickey DT (2014). A multilevel analysis of the effects of extrinsic rewards on motivation, engagement, and learning in elementary school students.Computers and Education, 75,136-148

Freedman, E., Hickey, D., Harris, T., Chartrand, G. Schamberger, B., and Luo, Q.M. (June 2023). Diversity, equity and inclusion are supported through productive disciplinary engagement and a broad environment. In J. Slotta, L. Charles. A. Breulleux, T. Laferriére and R. Casidy (editors),Build knowledge and nurture our community.Proceedings of the Annual Meeting of the International Academy of Learning Sciences. Montreal Kanada.

Gee, J.P. (1999). The Future of the Social Turn: Social Thinking and the New Capitalism.Research on language and social interaction,32(1-2), 61-68。

Glazewski, K.D., & Ertmer, P.A. (2020). Supporting diverse learners in solving complex problems: A spirit of intentionality for equal access.Research and development of educational technologies,68, 679-702。

JG Greenow (1997). Answer the wrong question in the statement.educational researcher,26(1), 5-17。

JG Greenow (1998). The state of knowledge, learning and research.American psychologist,53(1), 5-30。

Greeno, J.G., & Moore, J.L. (1993). Situations and Symbols: Answer Vera and Simon.cognitive science, 17,48-55。

Hall, R., & Rubin, A. (2013). Here are five small gaps: The teaching and learning dilemma with traditional tariff structures. InJ. G. Greeno & S.V. Goldman (editors),Thinking practices in learning mathematics and science(S. 199-246). Routledge.

Harry. T., Freedman, E., Jongewaard, R., Hickey, D.T., and Schamberger, B. (2023, June). A framework for generative subject engagement and expansion as a basis for teacher education. In J. Slotta, L. Charles. A. Breulleux, T. Laferriére and R. Casidy (editors),Build knowledge and nurture our community.Proceedings of the Annual Meeting of the International Academy of Learning Sciences. Montreal Kanada.

Herrington, J., Reeves, T. C. und Oliver, R. (2014)real learning environment.jump for joy.

Hickey, DT (1997). Motivation and teaching perspectives of contemporary social constructivismeducational psychologist,32(3), 175-219。

Hickey, D. T. (2003) Participation and marginal non-participation: A prominent sociocultural model of achievement motivation.Elementary Magazine, 103(4), 401-429

Hickey, DT (2005). Levels, Representation and Iteration: A Design-Based Assessment Framework with Potential for Improving VET. In H. Gruber, C. Harteis, RH Mulder and M. Rehrl (eds.),Bridging individual, organizational and cultural perspectives on professional learning(pp. 327-344). chew

Hickey, D. (2015a). A framework for interaction in competency-based curricula.Educational career review.https://edtechbooks.org/-NhMT

Hickey, DT (2015b). Situational responses to formative assessment challenges.Evaluation in Education: Principles, Policy and Practice,22(2), 202-223。

DT Hickey (2022). Generative Disciplinary Engagement and Extended Frameworks: Randi Engle's Accidental Legacy. In M McCaslin & T Good (editors)Routledge Online Educational Encyclopedia.https://edtechbooks.org/-YvuV

Hickey, D.T. and Chartrand, G.T. (2020). Skills Discovery and Integration and Engagement: The Ideal Role of Web-Enabled Digital Signage.Education and Information Technology, 25(2), 943-956.

Hickey, D.T., Chartrand, G.T. and Andrews, CD. (2020). The Extended Framework as a practical theory for instructional design.research and development of educational technologies,68 (2), 751-782。

Hickey, D. T., Chartrand, G. T. und Harris, T. (2021, 24. April). A systematic review of research on productive disciplinary engagement and broader context Paper presented at the annual meeting of the American Educational Research Association.

Hickey D T, Duncan J, Gaylord C, Hitchcock C, Itow R and Stephens S (2020). gPortfolios: A practical approach to asynchronous online allocation.Information and Learning Sciences, 121(5/6), 273-283。

Hickey, D.T. and Filsecker, M.K. (2012). Participatory assessment of organizational research in educational video games and beyond. In K. Littleton, E. Scanlon and M. Sharples, (eds.)organize inquiry-based learning(S. 146-174).Taylor and Francis.

Hickey, DT, & Granada, JB (2004). The influence of sociocultural theory on our theories of engagement and motivation. In D. McInerney and S. Van Etten (eds.).Great theory revisited:Sociocultural Influences on Motivation and Learning, Volume 4, (pp. 200-223). Information Age Publications

DT Hickey und T. Harris (2020)Responsive Engagement and Virtual Student Assessment (REVLA).Open online self-paced courses for teachers in Google Classroom.https://bit.ly/GoogleREVLA

Hickey, DT & Harris, T (2021). Redesign online grading, assessment, and testing using geographic knowledge.distance learning, 42(2), 290-309。

Hickey, D.T., Honeyford, MA, & McWilliams, J.C. (2013) Participatory evaluation in a climate of accountability. In H Jenkins & W Kelly,Reading in Participatory Cultures: BlendingMoby DickIn English class(S. 169-184).Teachers College Press

Hickey, D.T., Ingram-Goble, A., and Jameson, E. (2009). Create assessments and assessment designs in a virtual learning environment.Journal of Science Education and Technology, 18,187-208

Hickey, D.T., Kindfield, AC, Horwitz, P. and Christie, M.A. (2003). Integrate curriculum, instruction, assessment, and assessment in a technology-enabled genetics environment.American Journal of Educational Research,40(2) 495-538。

Hickey, DT and Lam, C. (2022, April 24).Focus on metaphorical situations of culturally sustainable educational evaluationPaper presented at the annual meeting of the American Educational Research Association.

Hickey, DT and Lam, C. (2023, April 15).Culturally sustainable education assessment: A new approach to an enduring challenge.Paper presented at the annual meeting of the American Educational Research Association.

Hickey, D.T., & Lam, D. (in press). New perspectives on learning transfer. In A. O'Donnell & J. Reeve (eds.).Handbook of Educational PsychologyOxford University Press. Manuscript acceptance, June 14, 2021.

Hickey, DT, & McCaslin, M (2001). Comparative and socio-cultural background and motivation analysis. In S. Volet & S. Järvelä (eds.),Motivation in the learning context: Theoretical and methodological implications.(S. 33-56). Pergamon/Elsevier

Hickey, D.T., McWilliams, J.T. and Honeyford, M.A., (2011). Read Moby-Dick in Participatory Cultures: Organizational Participation Assessment in the New Media Age.Journal of Educational Computing Research, 44(4), 247-27

Hickey, D & Pellegrino, JW (2005). Theory, level and function. Understand the three dimensions of metaphors assessed by students. In J. Mestre (ed.)Transfer learning from a modern interdisciplinary perspective(pp. 251-293). Information Age Publications.

Hickey, DT, & Rehak, A. (2013). Wikifolios and participatory assessment of attendance, understanding, and performance of online courses.Journal of Educational Media and Hypermedia, 22(4), 229-263。

Hickey, D.T., Robinson, J., Fiorini, S., and Feng, Y. (2019). An internet-based alternative for fair preparation, access and success in portal courses.Internet in Higher Education, 1-14

Hickey, D.T. & Schafer, N.J. (2006). A design-based approach to participatory classroom management. In C. Evertson and C. Weinstien (eds.),Handbook of Classroom Management: Research, Practice and Current Issues(S. 887-908). Merrill-Prentice Hall

Hickey, D.T., & Schenke, K. (2019). Open digital signaling and reward structure. In KA Renninger & SE Hidi SE (eds.), The Cambridge Handbook of Motivation and Learning (pp. 209-237) Cambridge, MA: Cambridge University Press.

Hickey, D.T., Tassoobshirazi, G., Cross, D. (2012). Evaluateiflearn. Improve discussion, understanding, and performance in innovative science courses.Journal for scientific educational research, 49,1240-1270

Hickey, D.T. and Uttamchandani, S.L. (2017). Beyond Hype, Hyperbole, Myth and Paradox: Scaling Participatory Learning in Massive Open Online Courses. In L. Losh (Editor-in-Chief)The MOOC moment: Experimentation at scale and access to higher education(S. 13-36). University of Chicago Press.

Hickey, D. T., Uttamchandani, S. L. und Chartrand, G. T. (2020)Competence in context: New approaches to conceptualising, identifying and recognizing learning.In Bishop, M.J., Boling, E., Elen, J. und Svihla, V. (Hrsg.).Handbook of Educational Communication and Technology Research(547-592). jump for joy.

Hickey, D. T., Willis, J. E. und Quick, J. D. (2015).Where the signal is most effective.Introduction to the EDUCAUSE study program. https://edtechbooks.org/-QLku

Hickey, D.T., & Zuiker, S.J. (2012). A multi-level assessment of speaking, comprehension and performance in an innovative learning environment.Journal of Learning Sciences, 22,(4) 1-65

That RC (2018)Professional development is not a summer job: shaping for valuable and valuable teacher learning.PhD thesis. Indiana University.

Jenkins, H. (2012, March 4). How to Earn the "Mark" of the Defiants.Confessions of an Aca fan. Henry Jenkins official blog[Pause].https://edtechbooks.org/-RSVH

Koedinger, KR, McLaughlin, EA, and Heffernan, N.T. (2010). Online formative assessment and quasi-experimental assessment of instructional systems.Journal of Educational Computing Research,43(4), 489-510。

Lave, J. & Wenger, E. (1991).Positioning Learning: Legal Regional Engagement. Cambridge University Press.

JL Lemke (2000). Across Timescales: Artifacts, Activities, and Concepts in Ecosystems.Ideas, culture and activities,7(4), 273-290。

Lepper, M.R., Greene, D. and Nisbett, R.E. (1973). Sabotage of children's intrinsic interests with extrinsic rewards: A test of the "excessive justification" hypothesis.Journal of Personality and Social Psychology,28(1), 129-137

Lepper, M.R., Henderlong, J. and Gingras, I. (1999). Understanding the Effect of Extrinsic Rewards on Intrinsic Motivation Use and Abuse in a Meta-Analysis: A Review by Deci, Koestner, and Ryan (1999).psychological bulletin, 125, 669–676。

Lepper, M.R., & Malone, T.W. (1987). Intrinsic motivation and pedagogical effectiveness of computer-aided education. In R.E. Snow & M.J. Farr (Eds.), Aptitude, learning and Instruction: III. Synthesis and analysis of Affection Processes (pp. 255-286). Hillsdale, NJ: Alder Tree.

Messik, S. (1994). The interaction of evidence and outcomes in performance appraisal review.educational researcher,23(2), 13-23。

National Academies of Science, Engineering and Medicine. (2018).How people learn II: learners, environment and culture.National Academy Press.

National Research Council. (2000).How people learn: brains, minds. experience and education.National Academy Press.

National Research Council. (2001).Knowing What Students Know: The Science and Design of Educational Assessment.National Academy Press.

National Quality Standards (2018).The national standard for quality online instruction.Author.https://edtechbooks.org/-UgWk

JF Paine (2018).Strategies for implementing personalized learning with underdeveloped evidence and resources: a perspectiveDie RAND Corporation.https://edtechbooks.org/-CYuk.

Pane, J. F., Steiner, E. D., Baird, M. D., Hamilton, L. S. und Pane, J. D. (2017)Information Progress: Information on personalized learning delivery and outcomes. Research paper.Die RAND Corporation.https://edtechbooks.org/-NYKV

Paris, D. (2012). Cultural pedagogy: Necessary changes in attitudes, terminology and practice.educational researcher,41(3), 93-97。

JW Pellegrino (2002). Know what students know.scientific and technical questions,19(2), 48-52。

Popham, W.J. (1990). Face Validity: A Siren's Call for Teacher Testers. In W.C. Conoley, J.Y. Mitchell Jr., S.L. Wise and B.S. Plake (eds.). Evaluation in teaching: purpose, practice and implications for the profession (pp. 1-14). Erbaum

Reeves, T. & McKenney, S. (2020)。forward.In M.J. Bishop, E. Boling, J. Elen, J. und V. Svihla (Hrsg.).Handbook of Educational Communication and Technology Research, Fifth Edition(pages v–x). jump for joy.

Resnick, M. (2012, February 27). Another badge skeptic [blog post].Histar.https://edtechbooks.org/-LiQH

Ruiz-Primo, M.A., Shavelson, R.J., Hamilton, L. and Klein, S. (2002). Evaluation of systemic scientific educational reforms: In search of pedagogical sensitivities.Journal for scientific teaching research,39(5), 369-393。

Ryan, R.M., & Deci, E.L. (1996). When Paradigms Collide: Comments on Cameron and Pierce's Claim that Rewards Do Not Destroy Intrinsic Motivation.Review of educational research,66(1), 33-38。

Sivan, E. (1986). Motivations for the social constructivist theory.educational psychologist,21(3), 209-233。

Sturgis, C. and Casey, K. (April 2018).Equal Design: Ensure the success of all students with competency-based education.Aurora-Institut.https://edtechbooks.org/-LeVm

Tang, S.H., & Hall, V.C. (1995). The Excessive Justification Effect: A Meta-Analysis.Applied cognitive psychology,9(5), 365-404。

Thomas, MJ (2002). Learning in disjointed structures: online forum spaces.Journal of Computer Assisted Learning,18(3), 351-366。

Zusho, A. and Kumar, R. (2018). Critical Thinking and Future Directions in the Study of Race, Ethnicity, and Motivation [Special Edition].educational psychologist,53(2).