A breadth of topics are indicated and discussed with respect to the training and testing of artificial intelligence systems in simulations and game environments.

The study of animal behavior is multidisciplinary and includes the scientific fields of: ethology, behavioral ecology, evolutionary psychology, comparative psychology and, more recently, comparative cognition. Comparative cognition is the study of cognitive processes across all species of animals, including humans.

Psychometrics is a scientific field of study concerned with the objective measurement of skills, knowledge, abilities, mental capacities, mental processes, attitudes, personality traits and educational achievement. Psychometrics topics include the assessment of cognitive development and intelligence. Psychometric measurements can be made of non-human animals, humans and AI systems.

Collective intelligence is a shared or group intelligence that emerges from collaboration, collective efforts and competition. Collective intelligence can emerge from the interactions of multiple AI systems in simulations and game environments.

The field of artificial intelligence makes use of simulations and game environments to train and to test AI systems. Examples of such simulations and game environments include: the Arcade Learning Environment, the OpenAI Gym, the Behavior Suite for Reinforcement Learning, the Obstacle Tower, and the Animal-AI Environment.

Machine teaching is the control of machine learning. Machine learning algorithms define dynamical systems where states, or models, are driven by training data. Machine teaching designs optimal training data with which to drive learning algorithms to target models.

AI systems can be envisioned which accelerate and optimize the training and testing of other AI systems.

Intelligent tutoring systems are AI systems which provide personalized instruction to learners. Traditionally, the learners are human students. The techniques of intelligent tutoring systems, however, generalize to the training and testing of AI systems.

Computerized adaptive testing is a form of examination that adapts to the exhibited capabilities of examinees. Items to be administered to examinees depend upon the nature of or the correctness of examinees’ previous responses.

Interactive storytelling is a form of digital entertainment in which storylines are not predetermined. While authors may create any settings, characters and situations which a narrative must address, readers or players experience unique stories based upon their interactions with storyworlds.

Intelligent narrative technologies can be envisioned which generate dynamic narratives in simulations and game environments for the training and testing of AI systems. Such narratives would unfold based upon the behaviors exhibited by and the decisions made by AI systems.

Automatic item generation uses computer algorithms to produce items, the basic building blocks of exams, tests, questionnaires and other instruments of psychometric measurement.

Procedural content generation uses computer algorithms to produce elements of simulations and game environments. Procedurally-generated content could be puzzles, tasks, tests or other varied content useful for the training or testing of AI systems.

Item response theory is a paradigm for the design, scoring, analysis and evaluation of items, exams, tests, questionnaires and other instruments of psychometric measurement.

Content evaluation is the analysis and evaluation of procedurally-generated content and narrative elements, for instance in terms of their utility with respect to the training and testing of AI systems.

Can the internals of AI systems be inspected and monitored during training and testing or are such systems effectively black boxes?

If one can inspect and monitor the internals of AI systems, then metrics based upon their algorithms, e.g. deep reinforcement learning, could be obtained.

If, instead, AI systems are effectively black boxes, then such systems might be modeled as one models players or students. In this regard, models of AI systems undergoing training or testing would update based upon observations of their behaviors or decisions in simulations or game environments.

Event processing is the analysis of streams of events and the deriving of conclusions from them. This includes the processing of events which occur in simulations and game environments.

Psychometric measurements and other important metrics can be obtained from processing those event streams which originate in simulations and game environments during the training and testing of AI systems.

A breadth of topics were indicated and discussed with respect to the training and testing of artificial intelligence systems in simulations and game environments.

Artificial intelligence systems can be evaluated with respect to their performances on IQ tests. The training of such systems can be optimized via machine teaching techniques.

Psychometric artificial intelligence posits that artificial intelligence systems can be designed to exhibit intelligence by solving the same intelligence test items as humans. The same methodologies and techniques are of use for measuring both varieties of intelligence: natural and artificial. Advancements with respect to the measurement of one variety of intelligence entail advancements with respect to the measurement of the other.

Machine teaching is the control of machine learning. Machine learning algorithms define dynamical systems where states, or models, are driven by training data. Machine teaching designs optimal training data with which to drive learning algorithms to target models.

Machine teaching is also applicable to educational scenarios with human students. Machine teaching can optimize lessons to help students to achieve educational goals. If one can assume a cognitive learning model of a student, one can use machine teaching techniques to reverse-engineer the optimal training data.

Machine teaching algorithms can generate worked examples and test items for training human students and psychometric artificial intelligence systems. The algorithmic generation of items is referred to as automatic item generation. Items can be evaluated utilizing item response theory.

Beyond the measurement of intelligence, sequences of exercises can be designed to activate, strengthen, coordinate and integrate specific regions and processes of both varieties of cognition: natural and artificial. That which there is to be learned from the intersection of machine teaching and psychometric artificial intelligence will advance our understanding of the teaching of human students via personalized sequences of exercises and activities.

Artificial intelligence systems can be evaluated with respect to their performances on IQ tests. The training of such systems can be optimized via machine teaching techniques.

We can envision the utilization of psychometric and educational exercises and activities in the training of artificial intelligence systems. Suitable exercises and activities include: IQ test items, exercises and activities from textbooks, items from quizzes and exams, and items produced via automatic item generation.

Artificial intelligence techniques have long been applied to games and game playing.

Computer simulations, including simulations of realistic environments, can be of use for training artificial intelligence systems such as software agents, robots and autonomous vehicles.

In addition to their utility with respect to simulating environments, computer simulations can be of use to artificial intelligence systems for planning courses of action and imagining the results or consequences of actions.

We can envision the utilization of social simulation in the training of artificial intelligence systems. Such social simulations might resemble play or improvisational theatre, designed to develop the social cognition, theory of mind and mentalization capabilities of artificial intelligence systems.

Social simulations can also be of use to artificial intelligence systems for planning courses of action and imagining the results or consequences of actions.

Teacher-student architectures include two varieties of interacting artificial intelligence systems: teachers and students.

Typically, the students are machine learning systems and the teachers are responsible for tasks including selecting the appropriate data, loss functions and hypothesis spaces to utilize. Teacher-student systems can exhibit co-evolution, students and teachers improving together over the course of iterative scenarios.

Interactive narrative can build upon simulation and social simulation, utilizing digital directors, drama managers and/or experience managers to enhance the pedagogical value of scenarios with respect to the training of artificial intelligence systems.

Interactive narrative is well suited for teaching machine ethics to artificial intelligence systems.

Simulated learners can approximate the learning processes of humans to such a degree that instructional design and pedagogy can be advanced through their use.

Some technical topics are broached toward the crowdsourcing of dialogue system content and behavior.

The content and behavior of a dialogue system can be represented in a number of ways.

Firstly, the content and behavior of a dialogue system can be represented in programming language source code files. Collaborative authoring, in this case, is a matter of integrated development environments and source code repositories, version control systems.

Secondly, the content and behavior of a dialogue system can be separated from source code files as data stored in some data format or in a database. Collaborative authoring, in this case, could require custom software tools.

Thirdly, a number of services, cognitive services, can encapsulate the content and behavior of a dialogue system. Collaborative authoring, in this case, could require utilization of such services or related user interfaces.

Fourthly, the content and behavior of a dialogue system can be represented as a set of interrelated, URL-addressable, editable pages. Servers can provide content for a number of different content types, for example hypertext for content authors and other formats for dialogue system user agents. Server-side scripting can be utilized to generate pages and generated pages can contain client-side scripting.

Fifthly, the content and behavior of a dialogue system can be represented as a set of interrelated, URL-addressable, editable diagrams.

Sixthly, the content and behavior of a dialogue system can be represented in transcript form. Transcript-based user interfaces may resemble instant messaging applications, scrollable sequences of speech bubbles, with speech bubbles coming from the left and right sides, such that users can edit the content in dialogue systems’ speech bubbles. Users could opt to view more than plain text in speech bubbles. There could also be vertical, colored bands in one or both margins, visually indicating discourse behaviors, moves, objectives or plans which span one or multiple utterances.

Debugging dialogue systems is an important topic. Debugging scenarios include switching from interactions with dialogue systems to authoring processes such that dialogue context data is preserved.

Natural language generation can produce editable structured documents from the data stored in databases and knowledgebases. Generated content can contain, beyond natural language, data and program logic to facilitate the processing of constrained or unconstrained edits. Edits to generated content can result in changes to stored data.

Computer-aided writing can convenience content authors and assure quality. Software can, generally speaking, provide users with information, warnings and errors with regard to tentative edits. Software can support users including with regard to their spelling, grammar, word selection, readability, text coherence and cohesion. Software can measure the neutral point of view of natural language. Software can also process tentative edits with regard to their logical consistency with respect to data stored in databases and knowledgebases.

Exploration into the collaborative authoring and debugging of dialogue systems could result in new wiki technologies. Wiki dialogue systems could resemble spoken language dialogue systems with transcript-based user interfaces, users able to easily switch between dialogue-based interactions and the editing of dialogue system content and behavior.

Instructional design is the theory and practice of designing instructional experiences and encompasses the design of educational resources such as hypertext, images, animations, charts, graphs, infographics, audio, video, 3D graphics, computer simulations, games, learning objects, courses, textbooks and the contents of intelligent tutoring systems.

Diagrams are symbolic representations of information according to visualization techniques. Diagrams, conveying information visually, are a powerful and expressive medium. Diagram authoring software applications allow individuals without computer programming expertise to participate in the design of resources, the entry of data and the development of software.

Educational resources such as learning objects, courses, textbooks and the contents of intelligent tutoring systems can be represented as diagrams.

Crowdsourcing leverages the intelligence and wisdom of crowds toward the collaborative design of resources and the solving of problems.

The crowdsourcing of instructional design is facilitated by software applications for the collaborative authoring of extensible diagrams.

Relevant software systems include wiki software, collaborative document authoring software, integrated development environments, extensible workflow and diagram authoring software and version control systems.

Quality control is essential to crowdsourced endeavors. Learning management systems may interoperate with crowdsourced diagrams. Intelligent tutoring systems may draw dialogue, behaviors and other contents from crowdsourced diagrams.

With a model of quality – a definition of quality – quality control can be described as matters of quality assessment and quality assurance.

The quality of crowdsourced instructional design relates to the quality of other crowdsourced endeavors, such as encyclopedias, insofar as the veracity and factual accuracy of the resultant information is paramount. Also important to the quality of instructional design is its pedagogical efficacy. As education science is considered to be the study of improving the teaching process, quality with respect to crowdsourced instructional design can be phrased as a moving target.

The assessment of quality can utilize educational data mining, learning analytics, learner feedback and other techniques for obtaining educational metrics.

The assurance of quality is approached as a matter of computer-aided design. Software can apprise users of pertinent design constraints, educational standards and recommendations. Software can provide users with insights into what is working and why with respect to design elements, structures and patterns in the contexts of learning objectives, plans and strategies. Software can provide users with information, warnings and errors with respect to courses being designed and with respect to existing courses. Software bots can process collections of educational resources, for example performing verification and validation services.

We need to standardize models for the diagrams of educational resources such as learning objects, courses, textbooks and tutorial contents. Diagrams of such educational resources should interoperate with both learning management systems and intelligent tutoring systems.

We can envision educators and instructional designers utilizing diagram authoring software. Both real-time collaboration and larger-scale crowdsourcing scenarios should be considered with respect to such software.

One promising area of research and development is computer-aided crowdsourced instructional design which leverages the intelligence and wisdom of crowds toward the design of diagrams of educational resources while the recommendations of software tools and the insights of education science are brought to bear for quality assurance.

Another promising area of research and development is computer-automated instructional design which encompasses the synthesis, selection and sequencing of multimedia educational resources per learning objectives, plans and strategies. Approaches may utilize machine learning and discovery upon collections of educational resources, such algorithms informed by educational metrics.

Improving the modeling of exercises and activities is an important task, one which advances the theory and practice of intelligent tutoring systems. The variety of modeling discussed, herein, is domain-independent, spanning multiple courses and the development of cognition. The proper, rigorous modeling of exercises and activities is non-trivial, an inherently complex task; doing so involves the modeling of cognition, modeling cognitive processes for numerous, relevant paths of progression through the exercises or activities.

With properly modeled exercises and activities, possibly a standardized such modeling, intelligent tutoring systems could be described as querying repositories of modeled exercises and activities, modifying existing exercises and activities or generating new exercises and activities, as needed, and loading and interactively presenting sequences of such exercises and activities to students. Essentially, case-based reasoning.

Repositories of exercises and activities require a querying language, possibly a standardized such language. Such querying languages depend strongly upon the modeling of exercises and activities.

Repositories of exercises and activities are more than case-bases, in a number of ways, and require an update language with expressiveness for educational data mining, for providing educational-scientific measurements pertaining to the utility and quality of exercises and activities.

Repositories of exercises and activities are envisioned as interoperable with numerous intelligent tutoring systems and future versions thereof. The modeling of exercises and activities, then, must encompass the specific features and approaches of each contemporary intelligent tutoring system.

A number of approaches to the automatic generation of exercises and activities can be interoperable with repository architectures. While repositories may interface as collections of existing exercises and activities, some may, in response to some queries, generate new exercises and activities on the fly.

Learning objects and digital textbooks also contain exercises and activities. Software components can encapsulate learning objects and digital textbooks, or collections thereof, implementing the interfaces of repositories. That is, intelligent tutoring systems can utilize specific exercises and activities, e.g. from syllabi or course materials, and intelligent tutoring systems can query collections of learning objects and digital textbooks for exercises and activities.

In the upcoming years, we can envision artificial intelligence systems utilizing educational exercises and activities as training data.

Developmental cognitive neuroscience, educational neuroscience and cognitive modeling will continue to advance and curricula should expand to include computer-mediated exercises and activities designed to activate, strengthen, coordinate and integrate specific regions and processes of the developing human brain.

Exercises and activities for cognitive development and enrichment can be generated by computer technology. Both human- and machine-generated exercises can be sequenced and scheduled by computer technology.

A focus on cognitive development results in a variety of exercises and activities. Variety also arises from mixing exercises and activities from multiple courses during coursework. Variety and the strategic sequencing and scheduling of exercises and activities promote student engagement.

Student affect should be monitored during the performance of exercises and activities. So doing facilitates the strategic sequencing and scheduling of exercises and activities. Sequencing and scheduling can be responsive to student affect; for example, scheduling categories of exercises known to be enjoyable to specific students to elevate mood.

Cognitive load refers to the total amount of effort being used in the working memory. Advancements to student modeling and cognitive modeling will facilitate better estimation of cognitive load during the performance of exercises and activities.

Discussion of the topics of variety in the sequencing and scheduling of exercises and activities entails a consideration of multitasking and task switching during problem solving. Modeling multitasking and task switching facilitates smoother experiences during and transitions between groups of exercises and activities.

As developmental cognitive neuroscience, educational neuroscience and cognitive modeling continue to advance, so too will our knowledge of transfer of learning and the cognitive neuroscience thereof. Transfer of learning pertains to how learning resulting from one category of exercise or activity effects performance in another.

The objectives of longer-scale exercise and activity sequencing and scheduling include preparing students for coursework days, weeks, months or years in advance. Students can, for example, enjoy: exercises and activities with boxes for arithmetical values, preparing them for eventual curricular topics from algebra; exercises and activities involving 3D visuospatial reasoning, preparing them for eventual curricular topics from trigonometry and geometry; and various exercises and activities designed to introduce concepts from calculus.

Intelligent tutoring systems facilitate computer-aided coursework, computer-mediated exercises and activities, providing mixed-initiative tutorial dialogue, explanation, hints and encouragement.

Intelligent tutoring systems can be advanced in a number of ways.

Intelligent tutoring at scale involves the tutoring of large populations of students. Interesting areas of research and development include: learning from interactions with students to improve the quality of tutoring, educational experimentation, A/B testing and multivariate testing.

Intelligent tutoring systems can be more interoperable with learning objects, digital textbooks and courseware. Intelligent tutoring systems could, for example, utilize exercises and activities from learning objects, digital textbooks and courseware.

The integration of multiple domains facilitates the mixing and scheduling of exercises and activities from multiple courses.

Tutoring across schoolyears involves longer-term student modeling and longer-term planning and scheduling of exercises and activities.

Intelligent tutoring systems can generate and sequence exercises and activities for purposes of activating, strengthening, coordinating and integrating specific regions and processes of the developing human brain.

Intelligent tutoring systems can tutor groups of students simultaneously.