Edubiotik : Jurnal Pendidikan.
Biologi dan Terapan ISSN 2528-679X .
ISSN 2597-9833 .
Vol.
No.
November 2025, pp.
412 Ae 423 Available online at:
https://ejurnal.
id/index.
php/edubiotik Research Article Enhancing biology concept mastery and metacognitive awareness through problem-based learning with concept mapping in a coffee plantation area Suratno 1,a,*.
RafiAoa Sastra Ningrum 1,b.
Bevo Wahono 1,c 1 Department of Biology Education.
University of Jember.
Jember.
Indonesia Email : suratno.
fkip@unej.
id 1,a,*, 210210103040@mail.
id 1,b, bevo.
fkip@unej.
id 1,c * Corresponding author Article Information Article History:
Submitted: 2025-02-25 Revised: 2025-12-24 Accepted: 2025-12-27 Published: 2025-12-31 Keywords:
Cognitive.
concept mastery.
problem-based learning Publisher Biology Education Department Universitas Insan Budi Utomo.
Malang.
Indonesia
ABSTRACT
This study addresses empirical learning problems found in senior high schools located in coffee plantation areas, where students tend to show low mastery of biology concepts and limited metacognitive awareness due to conventional, teacher-centered instruction.
The study aims to examine the effect of ProblemBased Learning (PBL) integrated with concept mapping on studentsAo biology concept mastery and metacognitive awareness.
A quantitative approach with a quasi-experimental pretestAeposttest nonequivalent control group design was The research was conducted at SMA Negeri Jenggawah.
Jember Regency, involving 69 tenth-grade students, consisting of 35 students in the experimental class and 34 students in the control class.
Data were collected using a biology concept mastery test and a metacognitive awareness inventory adapted from Schraw and Dennison.
The data were analyzed using ANCOVA after fulfilling normality and homogeneity assumptions.
The results indicated that PBL integrated with concept mapping had a significant effect on studentsAo biology concept mastery, but did not significantly affect their metacognitive awareness.
These findings suggest that PBL with concept mapping is effective in improving conceptual understanding, although longer and more intensive implementation is required to optimally enhance metacognitive awareness.
This study recommends the integration of contextual environmental problems and structured reflective activities to support metacognitive development.
How to Cite Suratno.
Ningrum.
, & Wahono.
Enhancing biology concept mastery and metacognitive awareness through problem-based learning with concept mapping in a coffee plantation area.
Edubiotik : Jurnal Pendidikan.
Biologi Dan Terapan, 10.
, 412Ae423.
https://doi.
org/10.
33503/ebio.
Copyright A 2025.
Suratno et al.
This is an open-access article under the CC-BY-SA license INTRODUCTION Knowledge of a concept alone in the 21st century is not sufficient to prepare students to thrive in a demanding world.
Students should be able to deeply understand the material and apply it in various contexts (Rizkia et al.
, 2.
A lack of balance between students' attitudes and their knowledge without proper application and deep conceptual understanding, can hinder the achievement of optimal learning This is where the urgency of metacognition becomes highly significant, as metacognitive skills, edubiotik@uibu.
: https://doi.
org/10.
33503/ebio.
Edubiotik : Jurnal Pendidikan.
Biologi dan Terapan Vol.
No.
, pp.
412 Ae 423 namely the ability to observe, control, and evaluate one's own thinking processes, play a crucial role in helping students understand how they learn and how they can improve their learning strategies (Azizah et al.
, 2.
The urgency of metacognition is very important to be applied in various learning processes to help students develop conceptual understanding and solve problems (Suratno et al.
, 2.
Yasir et al.
stated that empowering the urgency of metacognitive awareness in the learning process is necessary for several reasons.
First, metacognition is closely related to the ability to face challenges in the learning process and supports students in problem-solving.
Second, metacognitive awareness is linked to cognitive control abilities, which maximize personal development, academic writing skills, and conceptual mastery.
The problem-based learning (PBL) model is a learning model that focuses on problem-solving In this approach, educators primarily act as mediators and facilitators, helping students actively construct knowledge (Yulianti et al.
, 2.
The PBL model presents problem situations related to authentic and meaningful daily life, thereby fostering students' curiosity to investigate related topics and solve these problems, and it is hoped that students can implement them in their lives (Hidayatullah et al.
This learning model is particularly effective in science-based learning (Suharyat et al.
, 2.
According to Siswanti & Indrajit .
, problem-based learning has a greater impact on students' academic achievement compared to conventional learning models commonly applied in schools.
This learning model is centered on addressing a problem, serving as both the starting point for orientation and the endpoint for acquiring and applying new knowledge or insights.
The PBL process incorporates the concept mapping to facilitate and enhance learning within the PBL model.
The structure of concept mapping allows information to be represented in an alternative form, specifically through graphical visualization (Immelman et al.
, 2.
According to Zumaria et al.
the use of concept mapping enables students to present information in a hierarchical structure while establishing relationships between concepts (Murdiyah et al.
, 2.
This technique is characterized by the inclusion of cross-links with the presence of related relationships between concepts across different segments or main ideas in the concept map.
This approach is designed to illustrate the relationship between complex concepts and their interconnections (Novak et al.
, 2.
Students' ability to understand and organize conceptual material through concept mapping is used to measure their metacognitive With this technique, students can integrate ideas more clearly, linking prior concepts to new ones and expanding their knowledge base (Wahab et al.
, 2.
Metacognitive awareness is an individual's ability to control their cognitive domain.
This awareness is essential for students to develop into self-regulated learners by setting goals and managing themselves throughout the learning process (Marhaendra et al.
, 2.
According to Schraw & Dennison .
, an instrument for assessing metacognitive awareness was developed using 52 questions that measure both knowledge about cognition and cognitive regulation.
Metacognition consists of two main components:
knowledge of cognition and cognitive regulation.
Metacognitive awareness helps students recognize what they do not understand and how they can regulate their learning process (FaAou et al.
, 2.
Each individual's strong understanding of the learning process has a significant impact on cognitive learning, especially in increasing the effectiveness of learning strategies (Safitri et al.
, 2.
The application of active learning during the learning process can help students achieve mastery of biological concepts and optimize their performance (Azizaturrizkina et al.
, 2.
Concept mastery is one of the indicators used to assess students' understanding of the material they have learned.
This mastery means that students can comprehend a learning concept both scientifically and theoretically, with applications in daily life (Silviani et al.
, 2.
According to BloomAos taxonomy, several indicators define Suratno et al.
Ae Enhancing biology concept mastery and metacognitive awareness through .
Edubiotik : Jurnal Pendidikan.
Biologi dan Terapan Vol.
No.
, pp.
412 Ae 423 concept mastery, including remembering (C.
, understanding (C.
, applying (C.
, analyzing (C.
, evaluating (C.
, and creating (C.
In other words, concept mastery reflects students' comprehensive ability, ranging from lower-order to higher-order thinking skills (Mahmudi et al.
, 2.
Natural resources in each region have their own characteristics according to the location and geographical conditions.
Jember Regency has quite high plant biodiversity, ranging from lowlands to mountain slopes (Firmansyah et al.
, 2.
This area is divided into several development areas, each of which has different characteristics and regional potential, both in terms of human resource potential, natural resources, and physical and institutional infrastructure that supports development.
Where in several areas according to the ecological aspect have different characteristics, and plants play an important role in the agribusiness sector.
Where residents will analyze the potential characteristics of a plant to improve and develop an agribusiness, for example coffee plantations (Mahanani et al.
, 2.
A study by Suratno & Kurniati .
found that students in coffee plantation areas in Jember have problem-solving abilities that do not reach the expert level.
This suggests that students in these regions generally have moderate to low academic performance.
Learning that is directly related to the environment, such as coffee plantation areas, helps students understand abstract and complex material more easily (Adinia et al.
, 2.
Students who attend school in coffee plantation regions tend to be less active in the learning process, which can lead to boredom and lower cognitive learning outcomes.
Their level of learning independence also tends to be low, requiring educators to provide guidance to foster scientific attitudes.
To meet the global challenges of the 21st century, collaborative learning models are among the approaches that should be implemented to enhance student learning outcomes (Pratiwi et al.
Collaborative learning incorporates real-life problems from the studentsAo local environment, enabling them to apply the knowledge they acquire to their surroundings.
Schools in coffee plantation areas can serve as learning resources for environmental studies and the biodiversity of living organisms.
Thus, this study explores the influence of the PBL model combined with the concept mapping on students' metacognitive awareness and biology concept mastery in senior high schools located in coffee plantation RESEARCH METHODS This study used a quantitative quasi-experimental design with a pretestAeposttest nonequivalent control group (Table .
The population comprised all tenth-grade students at SMA Negeri Jenggawah.
Jember Regency.
Normality and homogeneity tests were conducted prior to sampling, indicating that the data were normally distributed and homogeneous (Sig.
> 0.
Consequently, two classes were randomly selected as the experimental class .
omprising 35 student.
and the control class .
omprising 34 The experimental group received problem-based learning integrated with concept mapping, whereas the control group was taught using the discovery learning model.
Table 1.
Research Design Class Pretest Treatment Posttest Experimental Group Control Group Explanation: O1=Experimental class before treatment.
O2=Experimental class after treatment.
O3=Control class before treatment.
O4=Control class after treatment.
X=Experimental group using Problem-Based Learning (PBL) with Concept Mapping, and Y=Control group using Discovery Learning Suratno et al.
Ae Enhancing biology concept mastery and metacognitive awareness through .
Edubiotik : Jurnal Pendidikan.
Biologi dan Terapan Vol.
No.
, pp.
412 Ae 423 Problem-based learning (PBL) is a student-centered learning model that emphasizes contextual problem-solving as the starting point of learning.
In the process, students are required to think critically, collaborate, and actively construct their own knowledge.
Meanwhile, concept mapping is a visualization technique used to represent relationships between concepts in a systematic map, helping students to organize, identify, and integrate the information they learn more deeply and critically (Jamil et al.
, 2.
The integration of PBL and concept mapping creates a strong synergy in the learning process.
PBL fosters higher-order thinking and problem-solving skills, while concept mapping helps to structure and visualize the thinking process explicitly, making information easier to understand, retain, and transfer to new The research instruments consisted of a biology concept mastery test and a metacognitive awareness inventory.
The biology concept mastery test included 10 multiple-choice items and 5 essay items covering indicators of remembering, understanding, applying, and analyzing biodiversity concepts.
The metacognitive awareness inventory was adapted from Schraw and Dennison and consisted of 52 items measuring knowledge of cognition and regulation of cognition (Table .
Both instruments were validated by experts and tested for reliability before implementation.
Table 2.
Metacognitive Awareness Indicators Component Knowledge about cognition Regulation of cognition Indicator Declarative knowledge Procedural knowledge Conditional knowledge Planning Information management strategies Monitoring Debugging strategies Evaluation Questions 8 questions 5 questions 7 questions 10 questions 7 questions 6 questions (Schraw and Dennison, 1.
Students' mastery of biological concepts was measured through pretest and posttest scores, which were administered to both the experimental and control classes.
The pretest results showed that most students had not yet understood the basic concepts and levels of biodiversity in living organisms.
However, after the learning process took place, the posttest results showed a significant improvement, with students being able to identify and explain the concepts of biodiversity levels in living organisms.
The test consisted of 10 multiple-choice questions and 5 essay questions, based on the biology concept mastery indicators presented in Table 3.
Table 3.
Biology Concept Mastery Indicators Biology Concept Mastery Indicators Students are able to mention the concepts and levels of biodiversity of living things.
Students are able to identify and differentiate levels of biodiversity of living things.
Students are able to explain the factors that influence the level of biodiversity of living things.
Students are able to mention the types of ecosystems of the ecosystem level of biodiversity of living things.
Students are able to present concepts in various representative forms of biodiversity of living things.
Learning Indicators Identifying the properties of concepts Giving examples Apply concepts logically Classify objects based on certain properties Restate the concepts that have been received and Presenting concepts in the form of image representations, tables, graphs, and sketches for the control class Suratno et al.
Ae Enhancing biology concept mastery and metacognitive awareness through .
Edubiotik : Jurnal Pendidikan.
Biologi dan Terapan Vol.
No.
, pp.
412 Ae 423 Presenting concepts in the form of concept mapping representations for the experimental The research procedure began with administering a pretest to measure studentsAo initial biology concept mastery and metacognitive awareness.
Subsequently, the learning intervention was conducted over several instructional meetings.
In the experimental class, learning activities followed the PBL syntax:
problem orientation, problem investigation, concept mapping construction, presentation of solutions, and The control class followed the discovery learning stages.
After the intervention, a posttest was administered to both classes.
The data analysis technique for metacognitive awareness was conducted at the beginning and end of the learning process, simultaneously with the administration of the pretest and posttest.
The metacognitive awareness inventory was developed using a rating scale as shown in Table 4.
The results of the metacognitive awareness data were compared with the metacognitive awareness scoring criteria presented in Table 5.
Table 4.
Metacognitive Awareness Inventory Scoring Response Agree (Ye.
Disagree (N.
(Schraw and Dennison, 1.
Table 5.
Metacognitive Awareness Score Criteria Score Range 1 Ae 20 21 Ae 40 41 Ae 60 61 Ae 80 81 Ae 100 (Suratno et al.
, 2.
Score Category Not yet developed Still very risky Not so developed yet Strat growing Well developed Very well developed The collected data were processed and analyzed using the ANCOVA (Analysis of Covarianc.
test, by calculating studentsAo scores from the pretest and posttest phases.
The metacognitive awareness scores were obtained from studentsAo responses to the questionnaire, which was developed based on the metacognitive awareness indicators listed in Table 2.
The calculation of the scores was carried out as ycCycaycycaycnycuyceycc ycIycaycuycyce Score = ycAycaycuycnycoycyco ycIycaycuycyce x 100 Biology concept mastery was measured through studentsAo pretest and posttest results from both the control and experimental classes.
The scores were analyzed using the ANCOVA test.
The biology concept mastery scores were derived from studentsAo test performance, which was aligned with the indicators outlined in Table 3.
The method of score calculation is as follows:
ycCycaycycaycnycuyceycc ycIycaycuycyce Score = ycAycaycuycnycoycyco ycIycaycuycyce x 100 Suratno et al.
Ae Enhancing biology concept mastery and metacognitive awareness through .
Edubiotik : Jurnal Pendidikan.
Biologi dan Terapan Vol.
No.
, pp.
412 Ae 423 Data analysis was conducted using ANCOVA to examine the effect of the learning model on the dependent variables, with pretest scores as covariates.
Prior to ANCOVA, normality and homogeneity tests were performed to ensure that the data met statistical assumptions.
FINDING AND DISCUSSION
Metacognitive awareness pretest and posttest scores can be seen in Table 6.
The ANCOVA results for metacognitive awareness are presented in Table 7.
Based on the ANCOVA test results in Table 7, the obtained significance value .
) is 0.
This significance value .
is greater than 0.
050, indicating that H0 is accepted and H1 is rejected.
Therefore, it can be concluded that the problem-based learning model with the concept mapping technique does not affect the metacognitive awareness of high school students in coffee plantation area schools.
Table 6.
Metacognitive Awareness Pretest and Posttest Scores Number of Mean A SD of Metacognitive awareness Class Students Pretest Posttest Experimental 81 A 9.
43 A 8.
Control 56 A 9.
86 A 9.
Table 7.
ANCOVA Test Result of Metacognitive Awareness Source Type i Sum of Squares Corrected Model Intercept Pretest_Meta Class Error Total Corrected Total Data Discrepancy Mean Square Category Very well developed Very well developed Sig.
Concept mastery pretest and posttest scores can be seen in Table 8.
The results of the ANCOVA test for concept mastery can be seen in the table below.
Based on the ANCOVA test results in Table 9, the obtained significance value .
) is 0.
This significance value .
is smaller than 0.
indicating that H0 is rejected and H1 is accepted.
Therefore, it can be concluded that the problem-based learning model with the concept mapping has an effect on high school students' mastery of biological concepts in schools located in coffee plantation areas.
Table 8.
Concept Mastery Pretest and Posttest Scores Number of Mean A SD of Concept Mastery Class Students Pretest Posttest Eksperimen 26 A 16.
69 A 14.
Kontrol 57 A 13.
57 A 13.
Table 9.
ANCOVA Test Result of Concept Mastery Source Type i Sum of Squares Corrected Model Intercept Pretest_Concept Class Error Total Corrected Total Mean Square Data Discrepancy Suratno et al.
Ae Enhancing biology concept mastery and metacognitive awareness through .
Sig.
Edubiotik : Jurnal Pendidikan.
Biologi dan Terapan Vol.
No.
, pp.
412 Ae 423 The results of the ANCOVA analysis showed that the significance value for metacognitive awareness was greater than 0.
05, indicating that the PBL model integrated with concept mapping did not have a statistically significant effect on studentsAo metacognitive awareness.
This finding may be attributed to studentsAo relatively high initial metacognitive awareness scores, which limited observable improvement, as well as the short duration of the intervention.
Metacognitive skills generally require long-term and consistent practice to develop effectively.
In contrast, the ANCOVA results for biology concept mastery revealed a significance value below 05, indicating that the PBL model integrated with concept mapping significantly improved studentsAo mastery of biology concepts.
This result supports previous studies stating that PBL encourages meaningful learning by engaging students in authentic problem-solving activities.
Concept mapping further supports this process by helping students organize, visualize, and integrate biological concepts The integration of real environmental issues from coffee plantation areas into PBL activities provided contextual relevance, which enhanced studentsAo engagement and understanding.
Through concept mapping, students were able to connect abstract biological concepts with real-world phenomena, leading to improved conceptual clarity and retention.
The results of the metacognitive awareness test using ANCOVA indicate that the PBL model with concept mapping does not significantly affect the metacognitive awareness of high school students in schools located in coffee plantation areas.
This outcome is due to the fact that the experimental class implemented the PBL model, while the control class used the discovery learning model.
Both models are student-centered learning approaches, where students play an active role as the primary participants in the learning process.
They are encouraged to actively explore, analyze, and solve problems or questions.
The analysis of pretest and posttest data on metacognitive awareness shows no significant increase in students' metacognitive awareness scores.
This is likely because students already had a high average pretest score in the metacognitive awareness inventory.
Additionally, the metacognitive awareness inventory was completed online via smartphones, which reduced the effectiveness of the assessment.
Some students tended to respond based on theoretical knowledge or perceived correct norms rather than reflecting their real-life experiences.
Students with such characteristics often provide answers aligned with social expectations or academic standards, even though, in practice, they may not fully apply the chosen concepts in their daily lives.
The results of this study found that the PBL model had no impact on students' metacognitive Similar findings were reported by Lidia et al.
, .
, who stated that the use of PBL in biology learning did not significantly influence students' metacognitive awareness, despite the fact that PBL strategies are generally expected to enhance metacognitive awareness.
A deeper understanding and improved metacognitive awareness tend to emerge in problem-based learning environments that are implemented over a longer duration.
The syntax of the learning model requires time and continuity for its full effect.
According to the applied learning model, students will develop characteristics aligned with the intended learning objectives.
Student habits and motivation play a crucial role, even though problembased learning and module-based instruction allow for evaluation and improvement activities.
Joshi et al.
explored the development of meaningful learning through the strategic implementation of concept mapping, which facilitates students' metacognitive processes and enhances their problem-solving strategies.
Problem-based learning has been shown to influence metacognitive awareness, where strong metacognitive skills enable students to effectively solve problems.
Students are trained in metacognitive skills such as planning, implementation, and evaluation of solutions when analyzing and solving complex problems, rather than simply acquiring a better understanding of the Suratno et al.
Ae Enhancing biology concept mastery and metacognitive awareness through .
Edubiotik : Jurnal Pendidikan.
Biologi dan Terapan Vol.
No.
, pp.
412 Ae 423 material (Kusuma et al.
, 2.
Metacognitive awareness refers to an individual's understanding of their own thought processes, including knowledge of cognitive strategies and how they acquire and manage information (Asaidah et al.
, 2.
On the other hand, metacognitive skills focus on the application of this knowledge to improve learning effectiveness (Ermin, 2.
The analysis of each metacognitive awareness indicator shows a significant increase in the knowledge about cognition and regulation of cognition indicators.
Metacognitive awareness encourages students to set their own learning goals and determine the techniques they will use to achieve those goals during the learning process (Hamsyah et al.
, 2.
Metacognitive awareness has an important function in this process, because students must be aware of how they organize information, select relevant concepts, and build logical structures.
With the support of concept mapping, students can enhance their activity in exploring information from various learning sources.
Through concept mapping, students actively organize, connect, and represent relationships between concepts, requiring a deep understanding of the studied material (Wahab et al.
, 2.
The results of the hypothesis test on biology concept mastery using ANCOVA showed that the problem-based learning model combined with the concept mapping technique influenced the biology concept mastery of high school students in schools located in coffee plantation areas.
Therefore, it can be concluded that the implementation of the PBL model with the concept mapping technique has a significant effect on the biology concept mastery of high school students in schools in coffee plantation These results support the statement by Narsan .
, which states that the use of the PBL model is effective in improving students' mastery of biology concepts.
Learning through a problem-based model can enhance knowledge by involving the process of understanding a problem and encouraging students to solve it by finding appropriate solutions.
This concept explains that PBL is an active learning model that places students as the main agents in solving problems (Hidayatullah et al.
, 2.
Active.
PBL that helps students communicate and collaborate is supported by the use of the concept mapping technique, which, when based on a problem-solving model, assists students in mastering biological concepts and developing problem-solving skills (Adinia et al.
, 2.
The teaching materials required in PBL with the concept mapping technique function to train students' problem-solving skills and mastery of biology concepts in schools located in coffee plantation areas, in the form of student worksheets.
The worksheets used are based on the problem-solving model integrated with the concept mapping technique, where the worksheets present the core learning content, concept maps, and problems related to coffee plantations.
Through the concept mapping technique, students are trained to identify and organize conceptual ideas into patterns, as well as to extract the main ideas from the information they read and from observing their surrounding environment (Prayitno et al.
With the guidance of teachers who provide the worksheets, students are supported in identifying the main ideas from the information they engage with.
Using the concept mapping technique, students are trained to construct and develop concepts based on the key ideas derived from concept maps that address issues related to the levels of biodiversity in living organisms.
The PBL model with the concept mapping technique is a combination of a learning model and technique that emphasizes learning activities oriented toward real world problems and the surrounding This approach focuses on engaging students in problem solving activities to find solutions, where students construct concepts related to the subject matter through a series of problem solving processes (Silviani et al.
, 2.
The material used in this learning process centers on the levels of biodiversity in coffee plantation areas.
Students are presented with a problem at the beginning of the lesson, which prepares them to learn and solve it.
Supported by teachers who provide student Suratno et al.
Ae Enhancing biology concept mastery and metacognitive awareness through .
Edubiotik : Jurnal Pendidikan.
Biologi dan Terapan Vol.
No.
, pp.
412 Ae 423 worksheets, the goal is for students to identify the main ideas from the information they read.
Through the concept mapping technique, students are trained to formulate and develop concepts based on key ideas derived from concept maps that relate to issues surrounding levels of biodiversity in living organisms.
This information is presented in the form of a concept that illustrates the relationships between specific concepts (Amin et al.
, 2.
The implementation of the PBL model can encourage students to engage in independent learning, with the concept mapping technique serving as a useful tool to reinforce meaningful learning.
According to Murdiyah et al.
, concept mapping is a learning technique that supports active learning.
This technique helps students strengthen the structure of their ideas, correct misconceptions, and enhance Both problem-solving and concept mapping are essential cognitive skills required for effective learning.
According to Alt et al.
, the concept mapping technique also assists instructors in assessing what students understand and how they connect the material to the overall course objectives.
By organizing and structuring information through concept mapping, students are able to acquire knowledge and gain deeper insight into the subject matter.
Learning that has a direct connection to students' environment helps facilitate their understanding of concepts.
This supports the research by Jannah .
, which states that learning integrated with the coffee plantation ecosystem enhances students' concept mastery and provides indirect knowledge about biodiversity in the plantation area.
The PBL model with the concept mapping encourages and facilitates students in developing ideas and concepts through easily understandable concept maps.
By engaging with real-world environmental issues, students improve their problem-solving skills and develop a contextual understanding of real-life challenges.
This learning approach fosters flexible thinking and curiosity, enabling students to explore and analyze concepts more deeply (Zaidah et al.
, 2.
Through PBL, students are actively involved in identifying and solving environmental problems, which strengthens their critical thinking and problem-solving abilities in real-world contexts.
CONCLUSION
Based on the results of this study, it can be concluded that the PBL model with concept mapping techniques did not significantly influence the metacognitive awareness of high school students in schools located in coffee plantation areas.
However, the PBL model with concept mapping did have a significant impact on students' mastery of biology concepts in these schools.
ACKNOWLEDGMENT
The researcher would like to thank the Department of Biology Education.
University of Jember, and its staff who have provided facilities and support so that this research can be completed.
REFERENCES