Introduction
In a previous blog, I asked the concluding question:
- How will industry and jobs change in the co-pilot first world(where humans and AI work together and AI is the co-pilot)?
- What will that mean to pedagogy and evaluation?
In this blog, I explain how such a framework could be implemented through the use of prompt engineering for GPT to create a reflective, deeper format for learning. We are exploring some of these ideas in our AI courses at the University of Oxford, however all views expressed here are solely my own. We are also developing these ideas as a platform for our edtech start-up with Pinckney Benedict (Salooki).
The challenge is significant because if education were made affordable, high quality, inclusive and tailored, society could benefit at many levels. In many ways, this aspiration also takes us back to an older time. I read an interesting book called THE OXFORD TUTORIAL: ‘Thanks, you taught me how to think’ Edited by David Palfreyman. The Tutorial system is a unique phenomenon that characterises the teaching at Oxford and Cambridge. I like to use the same reflective style of teaching also.As per the book, the tutorial is much more than a group discussion - because the Tutorial system encourages the student to think for themselves.
However, such a system does not scale easily due to the need for individual attention. Similarly, in the previous blog, I spoke of the idea of the Inverse Bloom’s taxonomy. The idea is simple: i.e., flip the well-known Bloom's taxonomy and put creativity at the center of the learning process. Doing so changes the dynamic from “what you know” to “what you can apply”.
Image source: plpnetwork
Not many people object to putting creativity at the center of learning. However, like the Tutorial system, evaluating and scaling such creativity is an entirely different matter altogether.
The approach and steps
I describe below a mechanism for developing and scaling the Inverse Bloom’s taxonomy. We are exploring these ideas and comments are welcome especially from other educators.
To summarise the approach
- Bloom’s taxonomy, while originally designed for organising learning objectives, could be easily adapted for problem solving
- The artefact developed by the student for solving a problem could be in the format of a research paper
- It is possible to map the sections of a research paper to the Bloom’s taxonomy
- It is possible to create sections of a research paper using GPT via prompt templates,
- By working collaboratively in a team via prompts, but with an overall awareness of the whole process(Background of metacognition - wikipedia - metacognition and more details of metacognition in education - Vanderbilt metacognition), such a paper could be produced with the inverse Bloom approach . By this, we mean, you do not strictly consider the learning sections in a hierarchy, but rather you start wherever you like based on the prompts but you keep the overall structure in mind.
- The paper and the prompts become the submission - which can also be evaluated
Hence the steps for working are
- You explain the whole process above i.e. build in the idea of awareness and metacognition - so at each step - they see where they (or their group) are at
- Define the tools (essentially the prompts - you may need to create more)
- Understand the output - i.e. the format of a research paper alongwith the prompts
- Map and develop elements of your problem like you are developing a research paper through the use of prompts
- Map and develop elements of your learning objectives through through (Inverse) Bloom’s taxonomy through the use of prompts
- Develop the overall paper
- Always be aware where you are (hence the inverse bloom/ metacognition) are the key
Let us explore these ideas in more detail.
Analysis
The use of Bloom’s taxonomy for problem solving
Firstly, let's consider how Bloom’s taxonomy could be used for problem solving.
Bloom's taxonomy is a framework for organising learning objectives. As such, it is not originally designed for problem solving.
The taxonomy has six levels, each of which represents a more complex cognitive process. The levels are:
- Knowledge: Remembering facts and information.
- Comprehension: Understanding the meaning of information.
- Application: Using information in new situations.
- Analysis: Breaking down information into its component parts.
- Synthesis: Putting together different pieces of information to create something new.
- Evaluation: Judging the value of information or ideas.
While Bloom’s taxonomy is not primarily used for problem solving, it can be adapted to do so by providing a structure for thinking about and approaching problems. To use Bloom's taxonomy for problem solving, you can start by identifying the problem that needs to be solved. Then, you can use the taxonomy to help you think about the problem at different levels of complexity.
- For example, if you are trying to solve a math problem, you might start by identifying the relevant facts and information (knowledge). Then, you might try to understand the meaning of the problem (comprehension). Once you understand the problem, you can start to think about how to apply the information to solve it (application).
- As you work through the problem, you might need to break it down into smaller parts (analysis). This will help you to identify the key components of the problem and to develop a plan for solving it.
- Once you have a plan, you can start to put it together (synthesis). This will involve combining different pieces of information and ideas to create a solution to the problem.
- Finally, you need to evaluate your solution (evaluation). This will involve considering whether the solution is effective and whether it meets the requirements of the problem.
Mapping the sections of Bloom’s taxonomy to a research paper
As you can imagine, the use of Bloom’s taxonomy for problem solving maps neatly to a research paper
A typical research paper generally consists of the following sections:
- Title: The title should succinctly convey the main focus of the research.
- Abstract: A brief summary of the paper that highlights the research objectives, methodology, key findings, and conclusions.
- Introduction: This section provides background information, context, and motivation for the research. It states the research problem, research questions or objectives, and outlines the significance of the study.
- Literature Review: A comprehensive review of existing literature and previous studies related to the research topic. It highlights the gaps in knowledge or areas for further exploration.
- Methodology: This section describes the research design, data collection methods, and procedures used in the study. It should provide sufficient details for others to replicate the study.
- Results: The results section presents the findings of the study. It includes data, statistical analyses, figures, tables, or other visual representations that support the research outcomes.
- Discussion: This section interprets the results in the context of the research objectives and existing literature. It discusses the implications, limitations, and significance of the findings and may propose future research directions.
- Conclusion: A concise summary of the main findings, their implications, and the overall contribution of the study.
- References: A list of all the cited sources in a specified citation format (e.g., APA, MLA) to acknowledge the work of other researchers and provide readers with the means to access those sources.
- Appendices (optional): Additional supplementary material, such as survey questionnaires, interview transcripts, or detailed technical descriptions, that support the research but are not essential for understanding the main paper
How do levels of Bloom’s taxonomy match to the sections of a typical research paper
Bloom's taxonomy is a hierarchical model used to classify educational objectives and cognitive processes. While it is not directly aligned with the sections of a research paper, we can draw connections between Bloom's taxonomy levels and the different components of a typical research paper based on the cognitive processes involved. Here's a rough mapping:
- Remembering (Knowledge): This level involves recalling or recognizing information. It aligns with the background information provided in the Introduction section, where previous studies, existing knowledge, and relevant literature are summarized.
- Understanding (Comprehension): This level involves grasping the meaning and interpretation of information. It is reflected in the Literature Review section, where the researcher comprehends and synthesizes previous studies to identify gaps, contradictions, or areas of interest.
- Applying: This level involves using information in a new context or situation. In a research paper, this can be seen in the Methodology section, where the researcher applies specific research methods, techniques, or tools to collect data and conduct the study.
- Analyzing: This level involves breaking down complex information into smaller parts and examining the relationships between them. The Results section demonstrates the analysis of collected data, statistical tests, and the interpretation of findings.
- Evaluating: This level involves making judgments or assessments based on criteria and standards. The Discussion section showcases the evaluation of the research findings in light of the research objectives, existing literature, and theoretical frameworks. It includes critical analysis, interpretation, and assessment of the implications and limitations of the study.
- Creating: This level involves generating new ideas, combining information in innovative ways, or producing original work. While not directly linked to a specific section, the overall research paper can be seen as a creation that contributes new knowledge or insights to the field.
This idea fits in well with sections of research papers which can be generated by chatGPT
Implementation
The idea could work for any type of content
Hence, given a body of knowledge (war and peace) and given a scaffolding (prompt templates for creating a format like a research paper as per above), the paper and prompts become a submission format and can be evaluated (at scale) using the inverse Bloom’s taxonomy by understanding the overall approach
To recap, the steps for working are
- You explain the whole process above i.e. build in the idea of awareness and metacognition - so at each step - they see where they (or their group) are at
- Define the tools (essentially the prompts - you may need to create more)
- Understand the output - i.e. the format of a research paper alongwith the prompts
- Map and develop elements of your problem like you are developing a research paper through the use of prompts
- Map and develop elements of your learning objectives through through (Inverse) Bloom’s taxonomy through the use of prompts
- Develop the overall paper
- Always be aware where you are (hence the inverse bloom/ metacognition) are the key
Comments
Some notes and comments re the above approach
- Should every document created by this approach take this format i.e. should a letter to your mom contain a literature review? :) Of course not :) The format here lends itself to more complex artefacts in education which need some form of depth
- Do you always advocate an LLM within a closed community? - yes, this approach is hence different from a general purpose chatbot.
- What is the primary purpose of the LLM? We see the primary purpose of the LLM as augmenting human creativity.
- What types of submissions does this format suit best? Thesis, capstone, hackathon. This also fits well with students publishing papers to https://arxiv.org/ and inculcating values of research for students.
- Does reverse Bloom mean that we ignore the fundamentals? No. Rather it means , we start from where we know and what we are interested in but we keep the overall picture in mind. Hence, in this model, metacognition plays a key role.
Conclusion
In this post, we showed how prompts and GPT could play a role in education coupled with metacognition and the inverse bloom’s taxonomy. We could achieve goals of personalization and scale in learning and assessment. More interestingly, they take us back to a much older, reflective style of teaching and learning characterised by the Oxford tutorial. Comments welcome - especially from other educators. The views expressed in this blog are my own. It's also an idea we have been developing at our start-up Salooki.
As I write this, we see that "Harvard University embraces generative #AI in the classroom, adopting it as an official learning tool for its flagship coding course - Computer Science 50: Introduction to Computer Science (CS50) https://lnkd.in/eGDpwp-8. It is nice to see that other major universities are also adopting generative AI within their learning and development.
References
Image source: https://pixabay.com/illustrations/brain-thinking-thoughts-learning-8003819/
I used chatGPT in some background research for sections like “The use of Bloom’s taxonomy for problem solving”