In a working paper by the Institute of Policy Studies, Pang, Seah, and Wong (2019) conducted a survey to find out public perceptions on Singapore’s built heritage and landmarks and to “offer insights on public understandings of old buildings and landmarks in Singapore”. They used both qualitative and quantitative research methods in a study consisting of two phases, and performed focus group discussions, survey questionnaires, and face-to-face computer assisted personal interviews (CAPI).
In the first phase, researchers engaged fifty-one Singapore citizens in seven focus group discussions and looked at the “themes derived from the qualitative analysis” (Pang et al., 2019, p. 9). Enyioko (2019) states that focus group discussions are a qualitative research technique which “involves interviews and discussions with a group of people”. In a report on the focus group discussions, Seah, Hou, and Pang (2018) detailed the survey questionnaire given to participants. In it, section one asked participants if they were aware of the selected sites and answered either a “yes” or “no” which can be coded as “1” or “0”. Section two asked participants to rate the perceived importance of sites on a scale of one to three where one is “not important to me”, two is “neutral”, and three is “important to me”; Subsequently, the three answer options were coded as “-1”, “0”, and “1” respectively. Sections three and four required participants to answer which two sites they chose as the most important and least important sites, and state the reasons for selecting these sites.
Here, researchers posed predetermined questions to a selected sample of participants through surveys which are a form of quantitative research (Neuman, 2014, Chapter 10; Blackstone, 2012, Chapter 8). However, it is crucial to note that surveys can contain both quantitative and qualitative elements depending on how the questions are worded (Ponto, 2015). Evidently, the first IPS survey contained questions which captured answers in quantitative and qualitative forms conducted through focus group discussions that are predominantly a qualitative research technique.
In the second phase, researchers drew upon the Department of Statistics sampling frame. They conducted a “door-to-door quota sampling method” that controlled for ethnicity, gender, and age to select respondents. Here, researchers performed “face-to-face computer assisted personal interviews” with 1515 Singaporeans between the ages of 18 to 70 and gave them questionnaires containing five sections with closed and open ended questions. It can be said that the IPS researchers approached the study from a mixed method research approach.
Kong and Yeoh noted that there were few studies using “quantitative research methods such as surveys” to gather data from the public regarding heritage and conservation matters (cited in Pang et al., 2019). There are a few reasons why these research methods were used. The IPS researchers may have been prompted by the low number of studies conducted through surveys garnered from members of the public and hence used this method to produce insights. Additionally, they could have held focus group discussions in order to gather “opinions from the general public about heritage and conservation issues” (Pang et al., 2019). Above all, surveys enabled the collection of quantitative data which allowed researchers to apply multivariate regression to assess and attempt to predict outcomes. It is also much faster to conduct focus group discussions and face-to-face computer assisted interviews as compared to sending teams of researchers to perform door-to-door surveying of the same number of participants. Finally, focus group discussions and face-to-face interviews provide their advantages. Participants can clarify questions with researchers on the spot and further improve the validity of the results. Furthermore, participants are more inclined to share and speak up in a mutual-sharing manner.
In the study, researchers sought to measure and understand the concept of national identity and public opinion regarding Singapore’s built heritage. Pang et al. (2019) further examined the “relationship between heritage activism and citizens’ perceptions of national identity” and adds that “the two concepts are constantly being reproduced and redefined in various ways”. Here, researchers conducted literature reviews in order to understand built heritage and its definitions. They consulted academic references which defined heritage and its associated values (Pang et al., 2019, pp 6-9). Through that understanding, researchers were in a better position to set criteria and select a sample of sites that fit those definitions in order to create the questionnaire for respondents. As such, they could improve the face, content, and criterion validity of the study by defining the constructs for analysis used for the questionnaires.
Next, two independent coders identified common themes in the discussion transcripts. Independent parties can improve face validity while reducing error and bias in data interpretation to derive insights and constructs for measurement in the questionnaire during phase two of the study.
In the second phase, researchers improved the content validity of their study by posing a set of questions to respondents through face-to-face computer assisted interviews. Questions on a scale of 1 to 7 sought to measure the degree to which respondents rated their knowledge, memories, physical appeal, and importance of sites. Pang et al. (2019) notes that the “four domains of evaluation do not stand alone as pre-existing categories of thought, but are mutually influencing” and observed several factors that held greater weight than others in influencing public perceptions. This is evident of the four elements being able to measure public perceptions.
Lastly, quota based sampling improved criterion validity and representativeness. Researchers identified age as a unit of analysis and gathered data from respondents between the ages of 18 to 70. Respondents were segmented into three groups (senior, middle-aged, and youngest). Researchers then looked into predictors and successfully drew statistically significant model contributions through factor analysis and multivariate regression.
Still, the study contained potential weaknesses. First, it focused on the differences in age between respondents. Yet, researchers noted that “personal experience, social media, mass education, and commercial marketing” all affect perspectives on heritage (Pang et al., 2019). The study could be improved by including questions asking participants to state the channel where they first learned of surveyed sites. This question is crucial because social media, regardless of respondent age, may have instead played a pivotal role in public perception of heritage conversation for sites such as the Bukit Brown Cemetery (Pang et al., 2019, p. 19).
Another potential weakness lies in section one of the survey. This section asked respondents whether they were aware of a site or not and provided “yes” or “no” answers (Seah et al., 2018, p. 19). However, Neuman (2014, p. 231) states that “using only two choices creates a crude measure and forces distinctions into only two categories”. He further adds that more choices ought to be presented to respondents so that categories can be later collapsed after data collection. If only two categories of answers are collected, then the data cannot be improved and made more precise. Thus, researchers can ask participants to rate their awareness through “strong”, “moderate”, “weak”, and “no” categories for improved granularity. It is crucial to formulate better questions in the early stages of a study since this survey feeds into phase two. Providing more categories may also generate more insights on how the degree of awareness feeds into heritage and national identity.
Word count: 1194
Blackstone, A. (2012). Principles of Sociological Inquiry – Qualitative and Quantitative Methods. Saylor Academy. Retrieved from https://saylordotorg.github.io/text_principles-of-sociological-inquiry-qualitative-and-quantitative-methods/s00-license.html
Enyioko, N. C. (2016). The Nature and Essence of Scientific Research. SSRN Electronic Journal. Retrieved from https://doi.org/10.2139/ssrn.2735663
Neuman, W. L. (2014). Social research methods: Qualitative and quantitative approaches. Harlow, Essex: Pearson Education Limited.
Pang, N., Seah, C. S. P., Wong, K. L. (2019). SURVEY ON THE PERCEPTIONS OF SINGAPORE’S BUILT HERITAGE AND LANDMARKS. Institute of Policy Studies. Retrieved from https://lkyspp.nus.edu.sg/docs/default-source/ips/ips-working-paper-36_survey-on-the-perceptions-of-singapores-built-heritage-and-landmarks_final_for-upload.pdf
Ponto, J. (2015). Understanding and Evaluating Survey Research. Journal of the Advanced Practitioner in Oncology, 6(2), 168–171. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4601897/
Seah, Hou, Pang. (2019). STUDY ON THE PERCEPTIONS OF SINGAPORE’S BUILT HERITAGE AND LANDMARKS FOCUS GROUP DISCUSSIONS REPORT. Social Lab, Institute of Policy Studies. Retrieved from https://lkyspp.nus.edu.sg/docs/default-source/ips/report_nhb-fgds-interim-210618.pdf