Journal of Cleaner Production 193 (2018) 363e371

Contents lists available at ScienceDirect

Journal of Cleaner Production

journal homepage: www.elsevier.com/locate/jclepro

An analysis of the difficulties associated to sustainability insertion in

engineering education: Examples from HEIs in Brazil
~ ez a, O.L.G. Quelhas c,
I.S. Rampasso a, R. Anholon a, D. Silva b, R.E. Cooper Ordon
Walter Leal Filho d, *, L.A. Santa-Eula
lia e
a
Department of Manufacturing Engineering and Materials, State University of Campinas, Mendeleyev Street, 200. Campinas, Sa ~o Paulo, Brazil
b
Laboratory of Innovation Technology Applied, Faculty of Education, State University of Campinas, Bertrand Russell Avenue, 801 Campinas, Sa ~o Paulo, Brazil
c
Laboratory of Technology, Business and Environment Management. Federal Fluminense University, Passo da Pa tria Street, 156. Nitero
i, Rio de Janeiro,
Brazil
d
Hamburg University of Applied Sciences, Research and Transfer Centre "Sustainable Development and Climate Change Management", Ulmenliet 20, D-
21033 Hamburg, Germany
e 
Ecole de gestion, Universit
e de Sherbrooke, 2500, boul. de l'Universit
e, J1K2R1, Sherbrooke, Canada

a r t i c l e i n f o a b s t r a c t

Article history: This paper aims to understand how the difficulties associated with the inclusion of sustainability in
Received 21 March 2018 engineering education are related. From the literature review, eleven difficulties were compiled and
Received in revised form posteriorly, a panel of experts was conducted to divide them into two groups, namely “difficulties
8 May 2018
associated with structure and planning” and “difficulties observed in didactic practice”. These groups
Accepted 9 May 2018
Available online 10 May 2018
were used as a basis of a survey, involving Brazilian lecturers who work with sustainability in engi-
neering courses. The collected data were analysed through Structural Equation Modelling, using Partial
^ as de
Handling Editor: Cecilia Maria Villas Bo Least Squares method. The validation of the model was divided into nine different steps. A causal
Almeida relationship between the two groups was verified through the present research, that is, the greater the
difficulties associated with the structure and planning, the greater will be the difficulties observed in
Keywords: didactic practice. The results of this paper may be used by researchers in their future studies and by
Engineering education lecturers and coordinators as a guide to the inclusion of sustainability in engineering education.
Difficulties © 2018 Elsevier Ltd. All rights reserved.
Sustainability
Structural equations modelling

1. Introduction Glassey and Haile, 2012). According to Staniskis and Katiliu  te_
(2016), sustainability needs to be central to the design of educa-
Many Higher Education Institutions (HEI) are increasingly con- tional policies in HEI. It has been encouraged over the years through
cerned about the education of the future professionals who will international declarations for sustainable development in higher
work in the market. Contrary to what occurred a few decades ago, education.
merely technical training no longer meets the requirements stip- These declarations are characterized as landmarks for the in-
ulated by society; it is necessary to develop a professional that, in clusion of sustainability into higher education since they provide
addition to the technical content mentioned, also have a critical guidelines to be followed in order to conduct this inclusion. Agenda
sense regarding environmental and social aspects (Barba-Sa nchez 21 in Rio Declaration, the Ubuntu Declaration (2002), The Earth
and Atienza-Sahuquillo, 2017; Chan et al., 2017; Fan and Yu, 2017; Charter and, specifically for the Engineering Education for Sus-
tainable Development (EESD), and the Barcelona Declaration
(2004) stand out among these landmarks (Segalas et al., 2008). The
last declaration is particularly important due to the role of engi-
* Corresponding author.
neers in society. Faced with a wide variety of possible practical
E-mail addresses: [email protected] (I.S. Rampasso), rosley@fem.
unicamp.br (R. Anholon), [email protected] (D. Silva), [email protected]
applications, the concern about sustainability becomes even more
(R.E. Cooper Ordon~ ez), [email protected] (O.L.G. Quelhas), walter.leal2@ critical in engineering courses (Glassey and Haile, 2012; Ortega-
haw-hamburg.de (W. Leal Filho), [email protected] (L.A. Santa- Sanchez et al., 2018).
lia).
Eula

https://doi.org/10.1016/j.jclepro.2018.05.079
0959-6526/© 2018 Elsevier Ltd. All rights reserved.
364 I.S. Rampasso et al. / Journal of Cleaner Production 193 (2018) 363e371

The traditional teaching of engineering, with an exclusive focus 2. Background

on optimizing solutions from the economic point of view, is
changing. Nowadays, responsibility for the negative impacts 2.1. Sustainability in engineering education
generated by these solutions also includes social and environ-
mental aspects. This new concept of “engineering responsibilities”, According to Barcelona Declaration (2004)1, engineers must
which permeates different values towards sustainability, must be consider their role in society and the impacts of their activities on
included in the education of future engineers (Guerra, 2017; the environment. They also should contribute to improve social
Palacin-Silva et al., 2017; Staniskis and Katiliu  te,
_ 2016). The new conditions and use their skills for sustainable development. Engi-
generation of professionals should work towards sustainable neering education must be integrated with sustainability aspects.
development, that is, they must meet the needs of society without The development of an adequate curriculum for engineering
compromising the capacity of future generations to attend their education, that contemplates the sustainability concepts requires
necessities (Brundtland, 1987; Huge  et al., 2018; Soini et al., 2018). planning, time, the involvement of stakeholders (Ma €lkki and
Therefore, the improvement of the curricula of the engineering Paatero, 2015) and the greater challenge: a “paradigm shift in en-
courses is essential for universities to follow the evolution of the gineering” (Mulder, 2017, p. 1107). Debating and planning the most
market and thus better empower its students (Bussemaker et al., appropriate manner to insert these concepts increase the chances
2017). 
of success and contributes to the continuity of actions (Avila et al.,
Although desired, this new conception of “engineering re- 2017; Danos et al., 2014; Ma €lkki and Paatero, 2015). Leal Filho et al.
sponsibilities” is still being inserted in many educational in- (2016) corroborated with this point of view, arguing that the results
stitutions. From the three sustainability dimensions, the economic and benefits of the mentioned inclusion may be impaired by the
one has always been present in the mentioned courses; the envi- lack of a previous planning.
ronmental sustainability, for its turn, is getting more and more The existing literature about the inclusion of sustainability in
space (Edvardsson Bjo € rnberg et al., 2015; Fisher and McAdams, engineering education presents interesting reports that may
2015; Haase, 2014; Hamid et al., 2017); however, the social greatly contribute to the improvement of these courses
dimension still needs more attention, according to Edvardsson (Edvardsson Bjo € rnberg et al., 2015; Holgaard et al., 2016; Salvatore
€ rnberg et al. (2015).
Bjo et al., 2016; Svanstro € m et al., 2012). The following paragraphs
The difficulties to insert sustainability concepts in engineering synthesize some of these reports, presenting their main charac-
education have been pointed out for a long time. Curricular changes teristics and contributions.
are complicated and sustainable development is a complex issue In a survey conducted at eleven US universities, Salvatore et al.
(Elms and Wilkinson, 1995). Currently, there are still many chal- (2016) identified that only six of them had sustainability actions
lenges to be overcome to fully integrate sustainability into the in their courses. It is important to emphasize that these actions
curricula of engineering courses (Glassey and Haile, 2012; Holgaard were performed in a complementary way through the provision of
et al., 2016; Huge  et al., 2018; Mulder, 2017; Palacin-Silva et al., extra courses and not through the full integration of concepts in the
2017; Sivapalan et al., 2017). Examples of these difficulties are the great majority of disciplines.
lack of integration among the areas that compose the courses Specifically addressing one of the sustainability dimensions,
(Bussemaker et al., 2017; Fan and Yu, 2017), the low understanding Edvardsson Bjo € rnberg et al. (2015) reported the inclusion of social
of the importance of interdisciplinarity by teachers (Fan and Yu, responsibility at KTH Royal Institute of Technology, an important
2017) or aspects of education for sustainability in relation to technical university of Sweden. The authors emphasize the diffi-
educational theories of the affective domain (Shephard, 2008), culties observed to conduct this inclusion and they point out useful
among others. didactic tools as problem-based learning (PBL), or project-based
Against this background, this paper aims to analyse the diffi- learning (PjBL) as done by Leal Filho et al. (2016). The approaches
culties associated with the inclusion of sustainability in engineer- favours the integration of sustainable development into the engi-
ing education in Brazil and tries to confirm the hypothesis that the neering curriculum (Guerra, 2017; von Blottnitz et al., 2015), since
difficulties associated with structure and planning have a causal students have opportunities to develop critical thinking and
relationship with the difficulties observed in didactic practice. problem-solving skills through a self-directed study that requires
There is no article in the studied literature that statistically proves interdisciplinary knowledge (Guerra, 2017).
this causal relationship, this being the research gap identified by us. Holgaard et al. (2016) presented in their study two interesting
Additionally, it will also present the average intensity noted for cases of the attempt to insert sustainability in engineering educa-
each difficulty according to the perception of lecturers involved tion, one in Denmark (Aalborg University) and the other in
with sustainability initiatives in engineering courses. The main Australia (Royal Melbourne Institute of Technology - RMIT). For the
contribution, in turn, lies in the fact that the results presented here case of Aalborg University especially, they highlight the importance
can be used in future research or as a guide for lecturers and course of PBL for the success of the mentioned inclusion. For the case of
coordinators. RMIT, they emphasize the curriculum based on capability, in which
Besides this introduction, the article contains four further sec- the education is focused on the necessary capabilities for students’
tions. Section 2 is dedicated to the literature review about sus- professional life.
tainability in engineering education; to provide an overview of the Svanstro€ m et al. (2012) reported the inclusion of sustainable
sustainability inclusion into engineering courses and to present a development concept at Chalmers University of Technology, in
background of relevant aspects of this inclusion. Section 2 further Sweden. According to them, the mechanical engineering program
presents the difficulties associated with its inclusion. Section 3 of the University uses conceive-design-implement-operate (CDIO)
describes the methodology used, showing the research classifica- framework, which favours the mentioned inclusion. The CDIO is a
tion and the procedures that allow the replication of the study by model that aims to make engineering education better suited to the
other researchers. Section 4 presents the results of the statistical
treatment through Structural Equation Modelling, as well as the
debates related to them. Section 5 presents the conclusions of the
research. 1
http://www.iau-hesd.net/sites/default/files/documents/declaration_of_
barcelona_english.pdf.
 I.S. Rampasso et al. / Journal of Cleaner Production 193 (2018) 363e371 365

demands of the job market. adequate and fully integrated didactic materials that allow a broad
Additionally, an important contribution to be mentioned is a vision of sustainability. In the same line of reasoning, Fenner et al.
theoretical study conducted by Mulder et al. (2010), through a re- (2005) comment that the few existing materials become rapidly
view of the literature about engineering education in sustainable outdated due to the rapid changes noted.
development. The authors provide relevant insights about aspects Still in relation to the interdisciplinary character of sustainabil-
of this inclusion, exploring the proper manner to conduct it. Envi- ity, in order to satisfactorily insert this issue into engineering
ronmental projects within the universities’ campi and problem- courses, it is important that universities are able to develop lec-
based learning are among these manners. However, they high- turers with new mentalities. These lecturers need to learn more
light the need for a cultural change to transform this inclusion in a about sustainability dimensions and work collaboratively with
concrete and lasting transformation. their co-workers (Iyer-Raniga and Andamon, 2016). Although this is
Although there is research in Brazil on the subject, the inclusion the desired situation, Mulder et al. (2012) emphasize the resistance
of sustainability in engineering education requires greater visibility of lecturers as an obstacle to achieving better results from the in-
in the country. Since the 1980s, sustainability at different levels of clusion of sustainability in engineering courses.
education is being inserted into Brazilian legislation. According to Another barrier pointed out by Biswas (2012) to obtain better
law no. 6938/81, environmental education should be present at all results is associated with the lack of interest of engineering stu-
levels of education. The 1988 Constitution reinforced this require- dents in relation to sustainability concepts, highlighting, in partic-
ment. In 1996, the Law of Guidelines and Bases of the Brazilian ular, the social aspects. There are many reasons for it, according to
Education nº. 9394/96 placed environmental education as a the authors, who emphasize the students' lack of maturity to deal
guideline for education in Brazil. However, only since 1999 e with with the intrinsic complexity of the theme of sustainability. These
law no. 9795/99, which established the National Policy on Envi- authors also point out that during graduation, students are usually
ronmental Education e that this inclusion has become more more interested in their grades than in the possible applications of
effective. Regarding engineering education, researchers point out concepts taught throughout their careers. It should also be noted
that in the decade of 2010 the aspects related to sustainable that in some cases the disinterest may be due to the lack of align-
development are still not properly inserted in undergraduate ment between what is taught in universities about sustainability
courses, especially in technological courses, as engineering and what is demanded by the market and by society (Hanning et al.,
(Loureiro, 2015). 2012; Sivapalan et al., 2017).
Finally, although Segalas et al. (2009) argue that there is no Returning to lecturers, the lack of motivation and their insecu-
standard manner by which sustainability should be embedded in rity may also be characterized as a hindrance and hamper the
higher education, the academic community still has much more to achievement of better results. Specifically, in relation to social
discuss on this issue, since the understanding of the difficulties sustainability, the lack of a clear definition of its meaning generates
associated with each case are important elements for maturing the insecurity in some lecturers during debates with their students
ideas (Tejedor et al., 2018). In this sense, the difficulties associated (Edvardsson Bjo €rnberg et al., 2015). In addition, some teachers also
with the inclusion of sustainability in engineering education will be fear losing autonomy by inserting concepts of sustainability in their
discussed in the following item. disciplines and acting in a collaborative way with other lecturers
(Bryce et al., 2004; Sivapalan et al., 2017). There is still a perception
that, when acting in isolation in their disciplines, professionals
2.2. Difficulties associated with the inclusion of sustainability in dominate a certain area of knowledge.
engineering education Some authors as Guerra (2017), Hopkinson and James (2010),
Schneider et al. (2008), Shields et al. (2014), and Sivapalan et al.
To analyse the main difficulties associated with the inclusion of (2017) also mention that the overload of existing disciplines in
sustainability in engineering education, it is important to define the engineering courses may be an obstacle to the inclusion of sus-
concepts of multidisciplinarity, interdisciplinarity and trans- tainability in the curriculum. Many lecturers need to work in
disciplinarity. In multidisciplinarity, a certain challenge is split into different courses and the lack of time prevents them from devel-
different parts and people from distinct areas work together to oping new ideas, preparing new materials or improving existing
solve it, with different focuses. In transdisciplinarity, a holistic disciplines (Desha and Hargroves, 2010).
approach is sought, an approach that crosses the limits of the Besides the teachers' lack of time, the scarcity of resources and/
knowledge areas, allowing full integration of different concepts. In or facilities to develop activities2 related to sustainability is
the interdisciplinarity approach, people from different areas work emphasized by several authors as a difficulty found when it aims to
together to create new knowledge that does not fit into any of the insert sustainability in engineering education (Iyer-Raniga and
original areas (Ashford, 2004; Guerra, 2017; Shields et al., 2014). Andamon, 2016; Rydhagen and Dackman, 2011; Sivapalan et al.,
Although these concepts were not created in the context of 2017). The availability of resources for the development of con-
sustainability education (Ertas et al., 2003), they have been widely tent related to sustainability is essential to train teachers and, thus,
mentioned by researchers from this area (Ashford, 2004; Guerra, enable them to master sustainable development concepts
2017; Jahn et al., 2012; Shields et al., 2014; Thompson et al., (Rydhagen and Dackman, 2011).
2017). In engineering courses, the difficulties for using the Logically, university commitment and constancy of purpose in
mentioned approaches seem to be greater. The curricula of the relation to activities linked to sustainability are important to ach-
engineering courses present very technical and relatively ieve better results, as mentioned by Rydhagen and Dackman (2011).
embedded disciplines in relation to the changes (Ashford, 2004; However, it is still observed in several universities a conservative
Desha and Hargroves, 2010; Martins et al., 2006; Mulder et al., vision, strongly associated to institutional processes that preclude
2012). Although they have to work in a complex global market, in the change towards the inclusion of sustainability in engineering
which sustainability stands out as an important multidisciplinary education (Bryce et al., 2004; Hopkinson and James, 2010). The
issue, engineering students are educated in courses with restricted
and isolated disciplines (Sharma et al., 2017).
For Nowotny et al. (2018), most of the difficulties associated 2
These activities may be any action or program carried out to promote sus-
with the adoption of interdisciplinarity are due to the lack of tainable development education.
366 I.S. Rampasso et al. / Journal of Cleaner Production 193 (2018) 363e371

challenges created for the inclusion of sustainability in engineering research, as Gray (2017) points out. Studying the difficulties asso-
courses by rigid institutional structures justify the need for flexi- ciated with the inclusion of sustainability in engineering education,
bility of these structures (Iyer-Raniga and Andamon, 2016). The we are studying educational phenomena and, therefore, we have a
teaching of concepts associated with sustainability should be qualitative approach. On the other hand, using a numerical scale to
considered a priority issue (Sivapalan et al., 2017) and, for this, quantify the degree of observation of each difficulty in the in-
universities must rethink their models and institutional processes stitutions' environment and to carry out statistical analyses, we
(Rydhagen and Dackman, 2011). performed a quantitative approach. Considering its nature, this is
Focusing on the traditional disciplines of engineering courses, it an applied research and in relation to its objectives, it can be
is observed that many of them are taught in an isolated way and, classified as exploratory. We understand that there is still much to
therefore, it is difficult to insert sustainability according to a be discussed with regard to the theme of inclusion of sustainability
transdisciplinary vision (Sivapalan et al., 2017). When taught in in engineering education. Finally, the instrument for data collection
specific disciplines, sustainability is generally approached in a was a questionnaire and the analysis of the data was done through
partial way with a greater focus on environmental aspects modelling of structural equations.
(Hanning et al., 2012). Besides, Sivapalan et al. (2017) point out the The following steps were taken to achieve the results: literature
difficulty of understanding the relationship between engineering review, a panel of experts to create a first theoretical model, a
and social and environmental sustainability. Social aspects, survey to collect data from lecturers who participated or still
particularly, are a great challenge for the teaching related to sus- participate in initiatives associated to the inclusion of sustainability
tainable development in engineering education due to lack of in the engineering courses and, finally, an attempt to validate the
clarity on this topic (Edvardsson Bjo€ rnberg et al., 2015; Hopkinson proposed model through modelling of structural equations. Each
and James, 2010). mentioned stage will be detailed below, allowing the replication of
Table 1 below summarizes the difficulties associated with the this research.
inclusion of sustainability in engineering education, collected from
the literature, discussed above.
3.2. Review of the literature

3. Methodology The systematic literature review aimed to find out the diffi-
culties associated with the inclusion of sustainability in engineer-
Firstly, this section presents the research classification and, ing education reported in the literature in order to perform a panel
subsequently, the developed methodological procedures. of experts with the results. In this subsection, these difficulties
compiled in Table 1 are related. The articles analysed were from the
3.1. Research classification bases Springer, Science Direct, Taylor and Francis and Emerald
Insight. The terms used for the search were “engineering educa-
From the point of view of research strategies, this study was tion”, “sustainability”, “sustainable development”, “difficulties”.
based on desk research, a panel of experts and a survey. The desk Other terms were added after some searching, such as: “chal-
research was used to raise the difficulties associated with the in- lenges”, “barriers” and “EESD”.
clusion of sustainability in engineering education, the panel of Only manuscripts that specifically mentioned difficulties in
experts allowed the division of the referred difficulties into two engineering courses were used. A table with sentences regarding
groups and, finally, the survey provided the understanding of lec- difficulties found within the articles was constructed to group
turers’ perception about the difficulties. A mixed approach to the similar difficulties. After analysing each of the articles, the diffi-
problem was used, with both qualitative and quantitative aspects of culties were grouped according to their similarities. In the end, 11

Table 1
Difficulties associated with the inclusion of sustainability in engineering education, collected from the literature.

Difficulties References

Difficulty in integrating disciplines and contents aimed at transdisciplinarity in (Ashford, 2004; Desha and Hargroves, 2010; Hopkinson and James, 2010;
teaching sustainability Schneider et al., 2008; Shields et al., 2014)
Difficulty to debate the inclusion of new activities related to sustainability (Crofton, 2000; Guerra, 2017; Hopkinson and James, 2010; Schneider et al.,
because many professionals believe that the curricula of engineering courses 2008; Shields et al., 2014; Sivapalan et al., 2017)
are overloaded
Lack of access to adequate and constantly updated didactic material that (Fenner et al., 2005; Nowotny et al., 2018)
contemplates all sustainability dimensions for engineering courses
Difficulty in debating economic and social aspects in engineering disciplines, (Edvardsson Bjo€rnberg et al., 2015; Guerra, 2017; Hanning et al., 2012;
with a focus on environmental sustainability Hopkinson and James, 2010; Schneider et al., 2008; Sivapalan et al., 2017)
Lack of alignment between what is taught in engineering courses about (Hanning et al., 2012; Sharma et al., 2017; Sivapalan et al., 2017)
sustainability and the real market needs
Difficulty in training lecturers for sustainability teaching (Iyer-Raniga and Andamon, 2016; Martins et al., 2006; Mulder et al., 2012)
Lack of interest of engineering undergraduate students for subjects related to (Biswas, 2012; Rydhagen and Dackman, 2011)
sustainability
Lack of motivation of lecturers for the inclusion of sustainability in the €rnberg et al., 2015; Fenner et al., 2005;
(Bryce et al., 2004; Edvardsson Bjo
engineering course Guerra, 2017; Iyer-Raniga and Andamon, 2016; Mulder, 2017; Mulder et al.,
2012; Rydhagen and Dackman, 2011; Schneider et al., 2008; Sivapalan et al.,
2017)
Lack of adequate facilities and/or resources to develop activities associated with (Desha and Hargroves, 2010; Iyer-Raniga and Andamon, 2016; Rydhagen and
sustainability Dackman, 2011; Sivapalan et al., 2017)
Difficulty in changing disciplines and/or implementing new practices for the (Iyer-Raniga and Andamon, 2016; Sivapalan et al., 2017)
teaching of sustainability due to the rigidity of institutional structures
Lack of support from university's top management and/or the establishment of a (Bryce et al., 2004; Holgaard et al., 2016; Hopkinson and James, 2010; Kamp,
broad program aiming at greater promotion of sustainability teaching 2006; Rydhagen and Dackman, 2011; Sivapalan et al., 2017)
 I.S. Rampasso et al. / Journal of Cleaner Production 193 (2018) 363e371 367

difficulties were obtained, as shown in Table 1. the SEM, the correct allocation of parameters in thematic con-
structs is analysed and, later, the causal relations between these
3.3. Panel of experts constructs are studied (Henseler et al., 2016; Xue et al., 2018). In
order to facilitate the presentation of the results, the validation of a
After listing the difficulties via literature review, a panel was PLS-SEM model was divided into nine different steps, based on the
held with experts in engineering education. This panel was considerations proposed by Hair et al. (2014) and Ringle et al.
composed of 4 lecturers of mechanical engineering, 2 lecturers of (2014). This sequence will be used to describe the results. The
production engineering, and 3 lecturers of chemical engineering. software used were G*Power and SmartPLS. These steps were also
These lecturers were selected based on their experience with sus- used in other research of the group to which the researchers
tainability teaching in engineering courses and they also have ar- belong.
ticles published on this subject. All the experts have more than ten The first step to take is the elaboration of a model, based on the
years of experience in this area and two of them are members of a existing literature and experts’ opinion, to be statistically tested.
PhD program in sustainable development. The panel also had the The second step is the definition of minimum sample size, through
support of a doctor in education to coordinate the meetings. This the software G*Power (Ringle et al., 2014). To calculate it, the
panel's objective was to stratify the difficulties into thematic con- following parameters should be used: F test for the test family;
structs, thus creating a first theoretical model. linear multiple regression, fixed model and R2 deviation from zero
It is important to highlight that the panel of experts is a quali- for the statistical test; a test power of 80%; 5% for the probability of
tative technique, that aims to solve a problem through opinions of error; and effect size of 15% (Hair et al., 2014).
experts in the approached area (Campos et al., 2010). In this study, The third step is characterized by the validation of the proposed
the panel was used in a preliminary phase, to develop the basis of a model and the application of Partial Least Squares Structural
model statistically validated subsequently (Pinheiro et al., 2013). Equation Modelling (PLS-SEM). For this, PLS Algorithm must be run,
For this study, the panel was realised in one meeting and the result using the software SmartPLS, with path weighting scheme for the
was reached in the third round of debate. weighting scheme; mean 0 and variance 1 for data metric; 300 as a
maximum number of iterations and abort criterion of 0.00001
3.4. Survey (Ringle et al., 2014).
In the sequence, it is possible to evaluate the convergent validity
With the first theoretical model, it was possible to structure a through the average variance extracted (AVE) (fourth step). To
questionnaire and use it as a basis for collecting data from lecturers validate this step, the AVEs must have 0.50 in all constructs (Ringle
who participated or still participate in initiatives associated with et al., 2014).
the inclusion of sustainability in engineering education, in Brazil. In the fifth step, the internal consistency is analysed with
The questionnaire used was composed of 18 questions, of which Cronbach's Alpha (CA > 0.60) and Composite Reliability (CR > 0.70)
11 were related to the difficulties and 7 related to respondents’ coefficients. This analysis is conducted to verify the data bias, that
information. For each of the difficulties, the participants were is, the accuracy of data (Hair et al., 2014). However, specifically for
required to indicate, on a scale of 0e10, how much they observed it SEM, composite reliability is better to evaluate the internal con-
in the mentioned initiatives. At extremes, note 0 indicated an un- sistency (Ringle et al., 2014).
observed difficulty while note 10 indicated a difficulty observed in The sixth step aims to verify the correct allocation of the vari-
an intense manner. ables. For this, Chin criteria (1998) is used. To be validated, the outer
In Brazil, all research involving the participation of human be- loading of each variable must be higher in its own construct (Ringle
ings, even in the character of opinion, must pass through the et al., 2014). In step seven, the evaluation is carried out over the
appreciation of an ethics committee in research. The research coefficients of determination (R2 values). Through this analysis, the
project and the questionnaire were submitted to the Research predictive accuracy of the model is measured (Hair et al., 2014).
Ethics Committee of the University where the research was con- Values of 2%, 13% and 26% characterize small, medium and large
ducted and was approved, allowing researchers to perform the effect, respectively (Ringle et al., 2014).
survey. The “Bootstrapping”, from SmartPLS, is used in the eighth step.
The questionnaire was sent to 821 lecturers of engineering and This technique enables the analysis of linear correlations and re-
112 answers were obtained, thus characterizing a return rate of gressions for p-values
0.05, with 5000 samples. To be validated,
13.64%. Regarding the institutions where the respondents experi- all the calculated values must be higher than 1.96. This result in-
enced the inclusion, 20.87% were state, 46.96% were federal, 27.83% dicates that the linear correlations and regressions are valid for at
were private and 4.35% of the respondents did not answer. Among least 95% of the cases (Ringle et al., 2014).
the seventeen different engineering courses in which the re- Finally, the last step is composed of redundancy (Q2) and com-
spondents taught, Mechanical Engineering, Production Engineer- monality (f2). This is analysed to verify the quality of structure
ing and Civil Engineering stood out. To facilitate data collection and adjustment (Ringle et al., 2014). The Q2 verify how much the model
tabulation, we chose to use the Google Forms platform. The ques- structure is close to what was expected of it. It should have values
tionnaire was available on the mentioned platform for a period of greater than zero. The Cohen Indicator (f2) shows the usefulness of
two months. each construct for the model and values of 0.02, 0.15 and 0.35 are
considered small, medium and large, respectively (Hair et al., 2014;
3.5. Data analysis Ringle et al., 2014). These values are obtained through the module
“Blindfolding” in SmartPLS.
The resulting survey database was used to analyse the model
proposed in the panel of experts and the attempt to validate this 4. Results and discussion
model was conducted using the Structural Equation Modelling
(SEM) technique, using the Partial Least Squares method (PLS). Hair 4.1. Panel of experts
et al. (2014, p. 15): “PLS regression is a regression-based approach
that explores the linear relationships between multiple indepen- The panel of experts aimed to analyse the difficulties compiled
dent variables and a single or multiple dependent variable(s)”. In from the literature and create a first model of causal relationship.
368 I.S. Rampasso et al. / Journal of Cleaner Production 193 (2018) 363e371

From the results of the literature review, the experts divided the
difficulties into two constructs. The establishment of only two
constructs, the name of each construct and the distribution of the
difficulties between them were made by specialists. The experts
allocated the difficulties into two thematic constructs that are
related to each other, namely: Construct 1 - “Difficulties associated
with structure and planning” and Construct 2 - “Difficulties
observed in didactic practice”. Tables 2 and 3 present the difficulties Fig. 1. Theoretical initial model proposed by the panel of experts (Source: Authors).
allocated in each construct. For the references of the difficulties,
vide Table 1.
Therefore, the first theoretical model to be tested was defined, in
which, the Construct 1 influence the Construct 2, as illustrated in
Fig. 1.
Fig. 1 shows that, according to the panel of experts, the diffi-
culties associated with structure and planning directly influences
the difficulties observed in didactic practice. This is the model to be
tested in the next section.
Fig. 2. Values obtained by applying the PLS-SEM Method (Source: Authors).

4.2. Statistical validation

Table 4
Once the model to be tested is defined, it is necessary to Quality Criteria for the analysed model (Source: Authors).
calculate the minimum sample size (step 2). For this, G*Power Constructs AVE Composite Reliability Cronbach's Alpha
software was used with the parameters recommended by Hair et al.
Pl 0.512 0.860 0.804
(2014). The results showed that a minimum sample of 55 re- Pr 0.574 0.869 0.813
spondents is necessary. Thus, the database with 112 respondents
was fully adequate to the analysis (test power of 98.22%).
In step three, PLS Algorithm was applied. The values obtained
variance extracted (AVE). As can be observed in Table 4, all the
with this method are presented in Fig. 2 and Table 4.
values are higher than 0.50, which indicate the satisfactory result of
The values presented in Fig. 2 have two meanings. The value
the model convergence. For the composite reliability (step five), all
0.736 is the path coefficient. It indicates a high causality between
the values are higher than 0.70 and the values for the Cronbach's
the constructs. The values between the constructs and their pa-
alpha are greater than 0.60, denoting the absence of biases as
rameters are the factorial loads. They show how well each difficulty
recommended by Ringle et al. (2014).
is associated with their construct. In this model, the proximity of
The cross-loadings were analysed in step six. This analysis aims
factorial loads does not allow comparisons.
to verify the best allocation of each variable (Chin, 1998). Using this
The fourth step was characterized by the analysis of average
analysis it was found that the greatest outer loads for the param-
eters occur in their own constructs and therefore the difficulties are
Table 2 correctly allocated.
Construct 1. Difficulties associated with structure and planning (Pl). Step seven analysed the coefficients of determination (R2
Pl1 Difficulty to debate the inclusion of new activities related to values). For the proposed model, it is necessary to calculate the
sustainability because many professionals believe that the curricula of coefficients of determination only for the endogenous construct, in
engineering courses are overloaded this case, the construct 2, about the difficulties observed in didactic
Pl2 Difficulty in training lecturers for sustainability teaching practice. The calculated value of R2 was 0.542 which means that the
Pl3 Lack of access to adequate and constantly updated didactic material that effect is large and satisfactory, according to Cohen (1988) concep-
contemplates all sustainability dimensions for engineering courses tion. For Ringle et al. (2014), this value denotes the quality of the
Pl4 Lack of adequate facilities and/or resources to develop activities
adjusted model.
associated with sustainability
Pl5 Difficulty in changing disciplines and/or implementing new practices for To verify the assumption that linear correlations and regressions
the teaching of sustainability due to the rigidity of institutional are valid for at least 95% of the cases, the resampling technique was
structures used (step 8). The t-student values observed between the param-
Pl6 Lack of support from university's top management and/or the eters and their constructs were much higher than 1.96 (the lowest
establishment of a broad program aiming at greater promotion of
sustainability teaching
value identified for difficulty Pl3 ¼ 5.025). The t-student value be-
tween the two constructs was also shown to be adequate (18.249).
Finally it analysed the indicators of redundancy (Q2) and com-
monality (f2). They are associated with the quality of the adjusted
Table 3
Construct 2. Difficulties observed in didactic practice (Pr). model. In Table 5, it is possible to verify that all values are adequate
(Q2 > 0; f2 z 0.35), as recommended by Ringle et al. (2014).
Pr1 Difficulty in integrating disciplines and contents aimed at
After analysing these nine steps, the proposed model is
transdisciplinarity in teaching sustainability

Pr2 Difficulty in debating economic and social aspects in engineering

disciplines, with a focus on environmental sustainability
Pr3 Lack of interest of engineering undergraduate students for subjects Table 5
related to sustainability Quality Criteria for the validated model (Source: Authors).
Pr4 Lack of motivation of lecturers for the inclusion of sustainability in the
Constructs Redundancy Communality
engineering course
Pr5 Lack of alignment between what is taught in engineering courses about Pl 0.324 0.324
sustainability and the real market needs Pr 0.281 0.365
 I.S. Rampasso et al. / Journal of Cleaner Production 193 (2018) 363e371 369

validated. According to which, the construct “Difficulties associated It is possible, with this validation, to recognize that the over-
with structure and planning” directly influences the construct loaded curricula of engineering courses (Guerra, 2017; Shields et al.,
“Difficulties observed in didactic practice”. 2014; Sivapalan et al., 2017), the lack of lecturers qualification (Iyer-
Additionally, Figs. 3 and 4 show the arithmetic means for each Raniga and Andamon, 2016; Mulder et al., 2012), the lack of an
difficulty. As it is shown, the means were between 4.29 and 5.83, adequate and updated didactic material (Nowotny et al., 2018), the
which indicates the relevance of every difficulty for the analysed lack of adequate facilities and/or resources (Iyer-Raniga and
sample. Andamon, 2016; Sivapalan et al., 2017), the institutional struc-
tures rigidity (Iyer-Raniga and Andamon, 2016; Sivapalan et al.,
2017), and the lack of support from university top management
5. Discussion (Holgaard et al., 2016; Rydhagen and Dackman, 2011; Sivapalan
et al., 2017), observed during the planning phase directly in-
It is important to highlight that PLS-SEM is a soft modelling fluences the sustainable teaching practice in engineering courses.
technique regarding distributional assumptions, however, it pro- As a consequence, there is lack of transdisciplinarity (Desha and
vides “very robust model estimations” and it is used in explanatory Hargroves, 2010; Hopkinson and James, 2010; Shields et al., 2014),
researches (Hair et al., 2014). Therefore, the validation of the model excess focus on environmental aspects (Edvardsson Bjo €rnberg
obtained in this study does not imply that no other model may be et al., 2015; Guerra, 2017; Sivapalan et al., 2017), students disin-
validated. And the validation of other models does not invalidate terest (Biswas, 2012; Rydhagen and Dackman, 2011), lack of moti-
this study since it is exploratory and was statistically validated vation of lecturers (Guerra, 2017; Mulder, 2017; Sivapalan et al.,
through PLS-SEM. 2017), and a lack of alignment between what is taught in engi-
neering courses about sustainability and the real market needs
(Sharma et al., 2017; Sivapalan et al., 2017).
The results presented corroborate the literature since they
reinforce the importance of adequate planning for the success of
curricular changes in favour of sustainability (Avila et al., 2017;
Danos et al., 2014; Ma€lkki and Paatero, 2015).
Regarding the implications of this study, it is important to note
that when the difficulties from the construct “Difficulties associated
with structure and planning” are solved before the implementa-
tion, the difficulties from the construct “Difficulties observed in
didactic practice” might be minimized. However, there is no guar-
antee that solving the difficulties from the second construct, the
difficulties from the first one will diminish. Therefore, the diffi-
culties associated with structure and planning must be considered
in the first place, in order to enable a successful inclusion of sus-
tainability in engineering education.

6. Conclusions

This research verifies that for the analysed sample the diffi-
culties in planning the inclusion of sustainability in engineering
education directly affect the difficulties found in didactic practice.
Again, the originality of the present study is emphasized by the lack
Fig. 3. Arithmetic mean for each difficulty from planning group (Source: Authors).
of studies that statistically prove this relationship.
The main contribution of this exploratory research is the sta-
tistically validated model. According to this model, it is important
for HEI to give a special attention in aspects of the structuring and
planning phase for the inclusion of sustainability in engineering
teaching, since the difficulties of this phase directly affect the dif-
ficulties faced during the didactic practice.
In practical terms, the results of this study show that when a
lecturer or an HEI intends to insert sustainability aspects into en-
gineering education, there is a basis that needs to be fulfilled. If it is
correctly performed, students will have a broader education and
the society will benefit.
The causal relationship mentioned above and the degree of
observation of each difficulty can be used by academic researchers
in the education area, as a starting point for future research, or by
lecturer/coordinators interested in improving their courses. Based
on the results presented here, lecturers/coordinators may give
greater attention to the planning phase, consequently, minimizing
directly the difficulties observed in didactic practice.
The main limitation of this research is due to its exploratory
Fig. 4. Arithmetic mean for each difficulty from didactic practice group (Source: nature. We understand that the theme “sustainability in engi-
Authors). neering education” still requires much debate and there are no
370 I.S. Rampasso et al. / Journal of Cleaner Production 193 (2018) 363e371

conclusions that can be generalized for all cases. However, the Hanning, A., Priem Abelsson, A., Lundqvist, U., Svanstro €m, M., 2012. Are we
educating engineers for sustainability? Int. J. Sustain. High Educ. 13, 305e320.
statistical validation of a causal model through Structural Equation
https://doi.org/10.1108/14676371211242607.
Modelling allows conclusions to be drawn for a considerable Henseler, J., Hubona, G., Ray, P.A., 2016. Using PLS path modeling in new technology
sample. A future research could also focus on the development of a Research : updated guidelines. Ind. Manag. Data Syst. 116, 2e20. https://doi.org/
model for the inclusion of sustainability in engineering education, 10.1108/IMDS-09-2015-0382.
Holgaard, J.E., Hadgraft, R., Kolmos, A., Guerra, A., 2016. Strategies for education for
based on the difficulties outlined on this paper. sustainable development e Danish and Australian perspectives. J. Clean. Prod.
112, 3479e3491. https://doi.org/10.1016/j.jclepro.2015.09.063.
Hopkinson, P., James, P., 2010. Practical pedagogy for embedding ESD in science,
References technology, engineering and mathematics curricula. Int. J. Sustain. High Educ.
11, 365e379. https://doi.org/10.1108/14676371011077586.
Ashford, N.A., 2004. Major challenges to engineering education for sustainable Huge, J., Mac-Lean, C., Vargas, L., 2018. Maturation of sustainability in engineering
development. Int. J. Sustain. High Educ. 5, 239e250. https://doi.org/10.1108/ faculties e from emerging issue to strategy? J. Clean. Prod. 172, 4277e4285.
14676370410546394. https://doi.org/10.1016/j.jclepro.2017.07.143.
 €
Avila, L.V., Leal Filho, W., Brandli, L., Macgregor, C.J., Molthan-Hill, P., Ozuyar, P.G., Iyer-Raniga, U., Andamon, M.M., 2016. Transformative learning: innovating sus-
Moreira, R.M., 2017. Barriers to innovation and sustainability at universities tainability education in built environment. Int. J. Sustain. High Educ. 17,
around the world. J. Clean. Prod. 164, 1268e1278. https://doi.org/10.1016/j. 105e122. https://doi.org/10.1108/IJSHE-09-2014-0121.
jclepro.2017.07.025. Jahn, T., Bergmann, M., Keil, F., 2012. Transdisciplinarity: between mainstreaming
Barba-Sa nchez, V., Atienza-Sahuquillo, C., 2017. Entrepreneurial intention among and marginalization. Ecol. Econ. 79, 1e10. https://doi.org/10.1016/j.ecolecon.
engineering students: the role of entrepreneurship education. Eur. Res. Manag. 2012.04.017.
Bus. Econ. 1e9. https://doi.org/10.1016/j.iedeen.2017.04.001. Kamp, L., 2006. Engineering education in sustainable development at delft uni-
Biswas, W.K., 2012. The importance of industrial ecology in engineering education versity of technology. J. Clean. Prod. 14, 928e931. https://doi.org/10.1016/j.
for sustainable development. Int. J. Sustain. High Educ. 13, 119e132. https://doi. jclepro.2005.11.036.
org/10.1108/14676371211211818. Leal Filho, W., Shiel, C., Paço, A., 2016. Implementing and operationalising inte-
Brundtland, G., 1987. Our Common Future, World Commission on Environment and grative approaches to sustainability in higher education: the role of project-
Development. Oxford University Press. oriented learning. J. Clean. Prod. 133, 126e135. https://doi.org/10.1016/j.
Bryce, P., Johnston, S., Yasukawa, K., 2004. Implementing a program in sustainability jclepro.2016.05.079.
for engineers at university of technology, sydney. Int. J. Sustain. High Educ. 5, Loureiro, S., 2015. Compete ^ncias para a sustentabilidade/desenvolvimento sus-
267e277. https://doi.org/10.1108/14676370410546411. tenta vel: um modelo para a educaça ~o em engenharia no Brasil. Santa Catarina
Bussemaker, M., Trokanas, N., Cecelja, F., 2017. An ontological approach to chemical Federal University.
engineering curriculum development. Comput. Chem. Eng. 106, 927e941. Ma€lkki, H., Paatero, J.V., 2015. Curriculum planning in energy engineering educa-
https://doi.org/10.1016/j.compchemeng.2017.02.021. tion. J. Clean. Prod. 106, 292e299. https://doi.org/10.1016/j.jclepro.2014.08.109.
Campos, R.T.O., Miranda, L., Gama, C.A.P., da, Ferrer, A.L., Diaz, A.R., Gonçalves, L., Martins, A.A., Mata, T.M., Costa, C.A.V., 2006. Education for sustainability: chal-
Trape , T.L., 2010. Oficinas de construça ~o de indicadores e dispositivos de lenges and trends. Clean Technol. Environ. Policy 8, 31e37. https://doi.org/10.
avaliaça ~o: uma nova te cnica de consenso. Estud. e Pesqui. em Psicol. 10, 1007/s10098-005-0026-3.
221e241. https://doi.org/10.12957/epp.2010.9029. Mulder, K.F., 2017. Strategic competences for concrete action towards sustainability:
Chan, C.K.Y., Fong, E.T.Y., Luk, L.Y.Y., Ho, R., 2017. A review of literature on challenges an oxymoron? Engineering education for a sustainable future. Renew. Sustain.
in the development and implementation of generic competencies in higher Energy Rev. 68, 1106e1111. https://doi.org/10.1016/j.rser.2016.03.038.
education curriculum. Int. J. Educ. Dev. 57, 1e10. https://doi.org/10.1016/j. Mulder, K.F., Segalas-Coral, J., Ferrer-Balas, D., 2010. Educating engineers for/in
ijedudev.2017.08.010. sustainable development? What we knew, what we learned, and what we
Chin, W.W., 1998. The partial least squares approach for structural equation should learn. Therm. Sci. 14, 625e639. https://doi.org/10.2298/TSCI1003625M.
modeling. In: Marcoulides, G.A. (Ed.), Modern Methods for Business Research. Mulder, K.F., Segala s, J., Ferrer-Balas, D., 2012. How to educate engineers for/in
Lawrence Erlbaum Associates, London, pp. 295e336. sustainable development. Int. J. Sustain. High Educ. 13, 211e218. https://doi.org/
Cohen, J., 1988. Statistical Power Analysis for the Behavioral Sciences, second ed. 10.1108/14676371211242535.
Psychology Press, New York. Nowotny, J., Dodson, J., Fiechter, S., Gür, T.M., Kennedy, B., Macyk, W., Bak, T.,
Crofton, F.S., 2000. Educating for sustainability: opportunities in undergraduate Sigmund, W., Yamawaki, M., Rahman, K.A., 2018. Towards global sustainability:
engineering. J. Clean. Prod. 8, 397e405. https://doi.org/10.1016/S0959-6526(00) education on environmentally clean energy technologies. Renew. Sustain. En-
00043-3. ergy Rev. 81, 2541e2551. https://doi.org/10.1016/j.rser.2017.06.060.
Danos, X., Barr, R., Go  rska, R., Norman, E., 2014. Curriculum planning for the Ortega-Sa nchez, M., Mon ~ ino, A., Bergillos, R.J., Magan~ a, P., Clavero, M., Díez-
development of graphicacy capability: three case studies from Europe and the Minguito, M., Baquerizo, A., 2018. Confronting learning challenges in the field of
USA. Eur. J. Eng. Educ. 39, 666e684. https://doi.org/10.1080/03043797.2014. maritime and coastal engineering: towards an educational methodology for
899324. sustainable development. J. Clean. Prod. 171, 733e742. https://doi.org/10.1016/j.
Desha, C.J., Hargroves, K. (Charlie), 2010. Surveying the state of higher education in jclepro.2017.10.049.
energy efficiency, in Australian engineering curriculum. J. Clean. Prod. 18, Palacin-Silva, M.V., Seffah, A., Porras, J., 2017. Infusing sustainability into software
652e658. https://doi.org/10.1016/j.jclepro.2009.07.004. engineering education: lessons learned from capstone projects. J. Clean. Prod.
Edvardsson Bjo €rnberg, K., Skogh, I.-B., Stro€mberg, E., 2015. Integrating social sus- https://doi.org/10.1016/j.jclepro.2017.06.078.
tainability in engineering education at the KTH Royal Institute of Technology. Pinheiro, J., de, Q., Farias, T.M., Abe-lima, J.Y., 2013. Painel de Especialistas e
Int. J. Sustain. High Educ. 16, 639e649. https://doi.org/10.1108/IJSHE-01-2014- Estrate gia Multime todos: reflexoes, exemplos, Perspectivas. Psico - Rev.
0010. Eletro ^nica PUC - RS 44, 184e192.
Elms, D., Wilkinson, D., 1995. The environmentally educated engineer. Focus on Ringle, C.M., Silva, D., Bido, D., 2014. Modelagem de Equaço ~es Estruturais com
fundamentals. Eur. J. Eng. Educ. https://doi.org/10.1080/03043799508928302. Utilizaç~ ao do Smartpls. Rev. Bras. Mark. 13, 54e71. https://doi.org/10.5585/
Ertas, A., Maxwell, T., Rainey, V.P., Tanik, M.M., 2003. Transformation of higher remark.v13i2.2717.
education: the transdisciplinary approach in engineering. IEEE Trans. Educ. 46, Rydhagen, B., Dackman, C., 2011. Integration of sustainable development in sanitary
289e295. https://doi.org/10.1109/te.2002.808232. engineering education in Sweden. Eur. J. Eng. Educ. 36, 87e95. https://doi.org/
Fan, S.C., Yu, K.C., 2017. How an integrative STEM curriculum can benefit students in 10.1080/03043797.2010.539678.
engineering design practices. Int. J. Technol. Des. Educ. 27, 107e129. https://doi. Salvatore, D., Ellis, N., Nesbit, S., Ostafichuk, P., 2016. Embedding sustainability
org/10.1007/s10798-015-9328-x. principles into engineering education. In: Leal Filho, W., Nesbit, S. (Eds.), New
Fenner, R.A., Ainger, C.M., Cruickshank, H.J., Guthrie, P.M., 2005. Embedding sus- Developments in Engineering Education for Sustainable Development, World
tainable development at cambridge university engineering department. Int. J. Sustainability Series. Springer International Publishing, Cham, pp. 261e272.
Sustain. High Educ. 6, 229e241. https://doi.org/10.1108/14676370510607205. https://doi.org/10.1007/978-3-319-32933-8_22.
Fisher, P.B., McAdams, E., 2015. Gaps in sustainability education. Int. J. Sustain. High Schneider, J., Leydens, J.A., Lucena, J., 2008. Where is “Community”?: Engineering
Educ. 16, 407e423. https://doi.org/10.1108/IJSHE-08-2013-0106. education and sustainable community development. Eur. J. Eng. Educ. 33,
Glassey, J., Haile, S., 2012. Sustainability in chemical engineering curriculum. Int. J. 307e319. https://doi.org/10.1080/03043790802088640.
Sustain. High Educ. 13, 354e364. https://doi.org/10.1108/14676371211262308. Segalas, J., Ferrer-Balas, D., Mulder, K.F., 2008. Conceptual maps: measuring learning
Gray, D.E., 2017. Doing Research in the Business World. SAGE Publications, London. processes of engineering students concerning sustainable development. Eur. J.
Guerra, A., 2017. Integration of sustainability in engineering education. Int. J. Sus- Eng. Educ. 33, 297e306. https://doi.org/10.1080/03043790802088616.
tain. High Educ. 18, 436e454. https://doi.org/10.1108/IJSHE-02-2016-0022. Segalas, J., Ferrer-Balas, D., Svanstro € m, M., Lundqvist, U., Mulder, K.F., 2009. What
Haase, S., 2014. Engineering students' sustainability approaches. Eur. J. Eng. Educ. has to be learnt for sustainability? A comparison of bachelor engineering ed-
39, 247e271. https://doi.org/10.1080/03043797.2013.858103. ucation competences at three European universities. Sustain. Sci. 4, 17e27.
Hair, J.F., Hult, G.T.M., Ringle, C., Sarstedt, M., 2014. A Primer on Partial Least Squares https://doi.org/10.1007/s11625-009-0068-2.
Structural Equation Modeling (PLS-sem). Sage Publications, Thousand Oaks. Sharma, B., Steward, B., Ong, S.K., Miguez, F.E., 2017. Evaluation of teaching
Hamid, S., Ijab, M.T., Sulaiman, H., Md Anwar, R., Norman, A.A., 2017. Social media approach and student learning in a multidisciplinary sustainable engineering
for environmental sustainability awareness in higher education. Int. J. Sustain. course. J. Clean. Prod. 142, 4032e4040. https://doi.org/10.1016/j.jclepro.2016.10.
High Educ. 18, 474e491. https://doi.org/10.1108/IJSHE-01-2015-0010. 046.
 I.S. Rampasso et al. / Journal of Cleaner Production 193 (2018) 363e371 371

Shephard, K., 2008. Higher education for sustainability: seeking affective learning 279e292. https://doi.org/10.1108/14676371211242580.
outcomes. Int. J. Sustain. High. Educ. 9, 87e98. https://doi.org/10.1108/ Tejedor, G., Segalas, J., Rosas-Casals, M., 2018. Transdisciplinarity in higher educa-
14676370810842201. tion for sustainability: how discourses are approached in engineering educa-
Shields, D., Verga, F., Andrea Blengini, G., 2014. Incorporating sustainability in en- tion. J. Clean. Prod. 175, 29e37. https://doi.org/10.1016/j.jclepro.2017.11.085.
gineering education. Int. J. Sustain. High Educ. 15, 390e403. https://doi.org/10. Thompson, M.A., Owen, S., Lindsay, J.M., Leonard, G.S., Cronin, S.J., 2017. Scientist
1108/IJSHE-02-2013-0014. and stakeholder perspectives of transdisciplinary research: early attitudes,
Sivapalan, S., Clifford, M.J., Speight, S., 2017. Engineering education for sustainable expectations, and tensions. Environ. Sci. Pol. 74, 30e39. https://doi.org/10.1016/
development: using online learning to support the new paradigms. Australas. J. j.envsci.2017.04.006.
Eng. Educ. 4952, 1e13. https://doi.org/10.1080/22054952.2017.1307592. von Blottnitz, H., Case, J.M., Fraser, D.M., 2015. Sustainable development at the core
Soini, K., Jurgilevich, A., Pietik€ ainen, J., Korhonen-Kurki, K., 2018. Universities of undergraduate engineering curriculum reform: a new introductory course in
responding to the call for sustainability: a typology of sustainability centres. chemical engineering. J. Clean. Prod. 106, 300e307. https://doi.org/10.1016/j.
J. Clean. Prod. 170, 1423e1432. https://doi.org/10.1016/j.jclepro.2017.08.228. jclepro.2015.01.063.
Staniskis, J.K., Katiliu
 te,
_ E., 2016. Complex evaluation of sustainability in engineering Xue, X., Zhang, X., Wang, L., Skitmore, M., Wang, Q., 2018. Analyzing collaborative
education: case & analysis. J. Clean. Prod. 120, 13e20. https://doi.org/10.1016/j. relationships among industrialized construction technology innovation orga-
jclepro.2015.09.086. nizations: a combined SNA and SEM approach. J. Clean. Prod. 173, 265e277.
Svanstro€m, M., Palme, U., Knutson Wedel, M., Carlson, O., Nystro €m, T., Eden, M., https://doi.org/10.1016/j.jclepro.2017.01.009.
2012. Embedding of ESD in engineering education. Int. J. Sustain. High Educ. 13,