RESEARCH ARTICLE


The Indonesian Version of the Liebowitz Social Anxiety Scale - Self Report (LSAS-SR-Indonesia): Psychometric Evaluation and Analysis Related to Gender and Age



Wilis Srisayekti1, *, Efi Fitriana1, Marisa F Moeliono1
1 Padjadjaran University, Faculty of Psychology, Jalan Raya Bandung-Sumedang km.21, Jatinangor-Bandung 45363, Indonesia


Article Metrics

CrossRef Citations:
2
Total Statistics:

Full-Text HTML Views: 2775
Abstract HTML Views: 404
PDF Downloads: 482
ePub Downloads: 189
Total Views/Downloads: 3850
Unique Statistics:

Full-Text HTML Views: 1554
Abstract HTML Views: 258
PDF Downloads: 399
ePub Downloads: 141
Total Views/Downloads: 2352



Creative Commons License
© 2023 Srisayekti et al.

open-access license: This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: https://creativecommons.org/licenses/by/4.0/legalcode. This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

* Address correspondence to this author at the Padjadjaran University, Faculty of Psychology, Jalan Raya Bandung-Sumedang km.21, Jatinangor-Bandung 45363, Indonesia; Tel: +62 22 7794126; E-mail: wilis@unpad.ac.id


Abstract

Aim:

This study aimed to examine the psychometric properties of the Indonesian version of LSAS-SR (LSAS-SR-Indonesia), and determine the analysis related to gender and age.

Background:

A previous study reported that social anxiety exists in Indonesia. The psychometric properties of the instruments were not reported because it was not the focus of the study. Further studies were recommended, and it was necessary to develop an instrument for social anxiety in Indonesian (Bahasa Indonesia). The Liebowitz Social Anxiety Scale (LSAS) is one of the most frequently used scales for assessing social anxiety. The psychometric properties of LSAS-SR-Indonesia must be assessed so that the instrument can be more properly used.

Objective:

This study examined the psychometric properties of LSAS-SR-Indonesia, including internal consistency and item discrimination, as well as validity (convergent, internal structure, factorial, and content). In addition to the psychometric properties, this study analyzed its correlation to gender and age.

Methods:

A total of 2074 participants from the general population were involved in the current cross-sectional study applying multi-stage cluster sampling. They lived in seven big cities in Indonesia, representing the major ethnic groups, i.e., Bandung, Denpasar, Banjarmasin, Makassar, Yogyakarta, Padang and Medan. The participants were 16-38 years old, and 73% of them were female. The internal consistency was analyzed using a coefficient of Cronbach’s alpha, and the item discrimination was examined by calculating the corrected item-total correlation. Convergent validity was analyzed using the correlation with the Brief Fear of Negative Evaluation Scale (BFNE) and the Taijin Kyofusho Scale (TKS) using Pearson Product Moment Correlation. Internal structure validity was examined using Pearson Product Moment Correlation between LSAS subscales (LSAS Fear and LSAS Avoidance) and their subscales (LSAS Fear Performance, LSAS Avoidance Performance, LSAS Fear Social Interaction, LSAS Avoidance Social Interaction). Factorial validity was used to analyze construct validity using Confirmatory Factor Analysis (CFA). Content validity was carried out using the evaluation of an expert panel. The analysis of gender and age was performed using factorial analysis of variance (ANOVA). All the statistical analysis was performed using the computer programs SPSS v.21 and LISREL v.10.3.

Results:

The LSAS-SR-Indonesia has good internal consistency. The majority of items demonstrate good discrimination. A good convergent validity is found in the total score and the two subscales. The internal structure is empirically proved. All factor models fit the data, and RMSEA, RMR, NNFI, CFI, and AGFI are in the fit range. The CFA demonstrates that all items in all models have a significant loading factor of 5%. The result of the content validity shows a high degree of relevance and representativeness, with an agreement of 60-100%. There are significant differences both in fear and avoidance across gender. However, there are no significant differences in fear and avoidance across ages.

Conclusion:

The LSAS-SR-Indonesia is a reliable and valid instrument to assess social anxiety symptoms in Indonesia. It can be used to identify and assist in the preparation for promotion and intervention programs to prevent the development of the symptoms.

Keywords: Liebowitz social anxiety scale-self report, Psychometric evaluation, Gender, Age, Indonesia, CFA, TKS.



1. INTRODUCTION

Social Anxiety Disorder (SAD, previously known as ‘social phobia’) is characterized by a marked fear or anxiety in social situations, where there is a potential for evaluation by others [1]. The individuals believe that their behavior will be perceived in such a way that makes them experience feelings of embarrassment or humiliation. They have fears of appearing anxious, such as shaking, blushing, sweating, or being perceived as incompetent [2]. These normal feelings for individuals experiencing social anxiety are not proportional to what may reasonably be expected [3]. The situation could lead to an avoidance of social situations, where the individuals would withdraw from the social situation [4]. SAD is a pathological form of social anxiety phenomenon. In the general population, social anxiety ranges along a continuum from no anxiety/ fear to a ‘normal’ level, and to the psychopathological extreme. SAD is at the extreme level, when the mechanisms of behavior and cognition are dysfunctional, resulting in a highly debilitating clinical disorder [5, 6].

SAD is reported to be one of the most common of all anxiety disorders [7]. The lifetime prevalence rate is the highest (12%), compared with the lifetime prevalence estimated for other anxiety disorders, such as generalized anxiety disorder (6%), panic disorder (5%), and posttraumatic stress disorder (PTSD) (7%) [8]. In 2013, this lifetime prevalence increased to 13%, with a 12-month prevalence rate of 8% [9]. SAD produces burdens and impacts professional and private life [10-12]; about 90% of sufferers have psychosocial impairment [13]. For individuals with social anxiety, thinking about social situations might be similarly debilitating as facing real social situations. It could similarly increase emotional arousal and might result in fear that could lead to behavioral inhibition and avoidance [11]. SAD has high comorbidity with other disorders, such as depression and alcoholism [14, 12]. Individuals with SAD perceive their quality of life as low [15].

A previous study reported that social anxiety was found in Indonesia [16]. This study examined social anxiety and taijin kyofusho, and their clinical relevance in Indonesia and Switzerland. The psychometric properties of the instruments were not reported because it was not the aim of the study. The study included only Cronbach's alpha coefficient of the scales. The recommendations pointed out that further studies on the topic of social anxiety in Indonesia would be beneficial. An instrument for social anxiety in Indonesian (Bahasa Indonesia) is then required, and the psychometric evaluation of social anxiety scales in Indonesian must be assessed in order that the instrument can be more properly used.

Some instruments measuring social anxiety have been developed, including the Social Phobia Scale (SPS), Social Interaction Anxiety Scale (SIAS) [17, 18] and Fear of Negative Evaluation Scale (FNE) [19, 20]. The Liebowitz Social Anxiety Scale (LSAS) [21] is one of the most frequently used scales for assessing social anxiety [22], for example [16, 23-30]. LSAS has been used in the clinical trials of medications for social anxiety disorder [30], in studies evaluating the efficacy of cognitive–behavioural treatments [31, 32], and in combination with both [33]. LSAS has been the most used scale in clinical studies [33].

LSAS was originally developed as a semi-structured clinical interview for assessing social phobia [21] and was designed to measure fear and avoidance in social and performance situations [21]. LSAS consists of 24 items, with 11 items addressing the social interaction situations and 13 items addressing the performance situations. Each situation has two 4-point Likert scales, one for the intensity of fear and the other one for the frequency of avoidance of the situation. The interviewer requested the individuals to rate their fear in every situation as well as their tendency to avoid those situations. LSAS has a good internal consistency (α = 0.81–0.96), as well as good discriminant and convergent validities in the total score and the two subscales [31]. A two-factor (fear and avoidance) and four-factor (social interaction, public speaking, observation by others, eating, and drinking in public) models were proposed [15, 31].

A self-report version of LSAS (LSAS-SR), which was established later, demonstrated strong psychometric properties. The studies aiming to examine the psychometric properties of LSAS-SR found good results in the internal consistency parameter, with Cronbach's alpha coefficients varying between 0.61 and 0.98 [16, 31, 34-43]. Previous studies showed that the LSAS-SR had good test–retest reliability [34, 37, 40, 43], and it had a high interrater agreement [43]. The internal structure of LSAS-SR was proven fit in some studies, supporting the utility of the questionnaire and providing a measure for a researcher as well as for clinicians [34, 36, 37, 40-42, 44-47]. Concerning the convergent validity of the LSAR-SR, the instruments used to perform this correlation varied according to the study [31, 32, 34-38, 40, 43]. The results demonstrated an acceptable fit, as shown in the following examples. In the category of [48], it was found a moderate to strong correlation using the Social Phobia Inventory (0.52–0.81), the Brief Social Phobia Scale (0.67-0.84), and a weak to a moderate relationship using the Beck Anxiety Inventory (0.21-0.57) [37], a moderate to strong correlation using the Social Phobia Anxiety Inventory (SPAI) (0.44–0.81) [34], a strong correlation using SPAI (0.70) and SPS (0.70) [32], a strong correlation using Social Interaction Anxiety Scale (SIAS) (0.62-0.70), a moderate to strong correlation using Social Phobia Scale (SPS) (0.47-0.60) [38]. Regarding the divergent validity of LSAR-SR, the correlations were performed using scales that evaluate depression and general anxiety symptoms. The previous results using Beck Depression Inventory (BDI) demonstrated a moderate to strong correlation (0.48-0.81) [34], a weak to moderate correlation (0.29-0.46) [38], a moderate to strong correlation (0.59-0.69) [32]. The results using the State-Trait Anxiety Inventory (STAI) found a moderate correlation (0.59 trait scale) and a strong correlation (0.78 state scale) [32]. Previous studies did not report the content validity of the LSAS-SR. The analysis of gender and age showed that there were significant differences across gender and age [41].

The LSAS-SR has been adapted and validated in French [40, 49], German [50], Hebrew [51], Japanese [45], Portuguese [36, 37, 44], Spanish [36, 52], and Turkish [43]. The adapted LSAS-SR in various languages showed that the scale had good internal consistency with Cronbach's alpha coefficients varied between 0.61 to 0.96 for the fear scale [36, 37, 40, 43, 52], between 0.63 to 0.95 for the avoidance scale [36, 37, 40, 43, 51, 52], and above 0.90 for total [43, 51]. The test–retest reliability of the adapted LSAS-SR in various languages supported the utility of the questionnaire with the Cronbach's alpha coefficients varied between 0.82 and 0.98 [40, 51, 52]. Concerning the convergent validity of the adapted LSAS-SR in various languages, the correlation was performed using various instruments according to the study, for example, Fear of Negative Evaluation (FNE) [40], Social Interaction Anxiety Scale (SIAS), SPS, and State-Trait Anxiety Inventory (STAI) [45], Social Anxiety Questionnaire (SAQ) [36], Social Phobia Inventory Fear (SPIF), Brief Social Phobia (BSP) [37] and Beck Anxiety Inventory (BAI) [37, 43]. The results were not consistent. A strong correlation for the fear scale (0.55-0.83) as well as for the avoidance scale (0.55-0.83) was found by the studies of [37, 45], and a weak correlation (0.25-0.31) was shown by the studies of [40, 43]. As for the divergent validity of the adapted LSAS-SR in various languages, a significant correlation was found using Beck Depression Inventory (BDI) [40, 45].

The current study aimed to examine the psychometric properties of the Indonesian version of LSAS–SR (LSAS-SR-Indonesia). Development of the LSAS-SR-Indonesia is necessary because the original scale [21] is an English version. English is not a language for everyday and formal situations in Indonesia. The LSAS-SR-Indonesia is then necessary so that the instrument can be used to assess social anxiety symptoms in Indonesia. Furthermore, this study analyzed its relation to gender and age. The psychometric evaluations included internal consistency and item discrimination.

2. MATERIALS AND METHODS

The Indonesian version of LSAS-SR was developed following ITC Guidelines for Translating and Adapting Tests [53], organized into six categories: pre-condition, test development, confirmation, administration, scoring and interpretation, and documentation. Pre-condition category highlights several decisions that have to be made before the translation process begins. It consists of (1) obtaining permission from the holder of the intellectual property rights related to the test, and (2) evaluating the definition and content of the construct measured by the test and the item content, for example, by recruiting experts for the construct measured who are familiar with the cultural groups being tested. The test development category focuses on the actual translation process using appropriate translation designs and procedures. This category provides (1) evidence that the test instructions and item content have similar meanings for the intended populations, for example, by conducting a small try-out of the translated version, (2) evidence that the item formats, modes of administration, and the procedures are suitable for the intended populations. It is to ensure respondents’ familiarity with item formats and test administrations, (3) collect pilot data using the translated test for the item analysis and reliability analysis. The confirmation category includes selecting a sample with certain characteristics and sufficient size that are relevant for the empirical analyses, such as reliability and validity analysis using Confirmatory Factor Analysis (CFA). The administration category consists of the preparation of the administration materials and the translated instructions, the answering mechanism, and how the answers will be scored. The scoring and interpretation category includes an interpretation of group score differences. The documentation category provides a technical manual that includes the translation process and the information for scoring and norming [53]. In the current study, the guidelines were carried out as follows:

First, in the pre-condition category, this study requested permission from Mr. Liebowitz to examine the psychometric properties of the Indonesian version of LSAS-SR (LSAS-SR-Indonesia). A panel session involving three experts (two psychologists and one psychiatrist) was conducted to evaluate the construct measured.

Second, in the test development category, the Indonesian version of LSAS-SR was developed using the translation-back translation procedure [54]. Two independent translators, who were psychologists and fluent in both Indonesian (Bahasa Indonesia) and English, translated the questionnaire from English to Indonesian. The panel consisted of both translators; the first and the third authors discussed the Indonesian translations and made the necessary corrections to the inconsistencies in the translations, resulting in the Indonesian version of the questionnaire. Two other independent translators, who were psychologists and fluent in Indonesian and English, translated the Indonesian version of the questionnaire from English to Indonesian. Using similar procedures, the panel consisted of both back-translators; the first and the third authors discussed the English back-translation and compared it to the original English version. There was no significant difference between the English back-translation version and the original English version. This procedure resulted in the pre-final Indonesian version of the paper-and-pencil LSAS-SR. The pre-final Indonesian version of LSAS-SR was then administered to limited participants, to ensure that the items and instructions of the questionnaire were understood and appropriate for the participants. The survey was carried out to 40 participants whose age range was similar to the main study’s participants’. In addition to questionnaire completion, they were requested to provide some inputs on the overall presentation of the questionnaires, including the items and scales, as well as any identified difficulties in completing the questionnaires. This procedure resulted in the final Indonesian version of the paper-and-pencil LSAS-SR. Concerning the examination of content validity, six psychologists were involved to examine the equivalence aspects of the questionnaire, namely semantic, idiomatic, experiential, and conceptual equivalence. They were invited to rate each item using 4-point Likert scale (1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = highly relevant), followed by a panel discussion [55].

Third, in the confirmation category, this study selected the participants using multi-stage cluster sampling following the procedure described in the section on participants to meet the required amounts [56]. The reliability of the questionnaire was examined using the coefficient of Cronbach’s alpha in order to analyze the internal consistency and item discrimination. The validity of the questionnaire was analyzed using convergent validity, internal structure validity, factorial validity, and content validity.

Fourth, in the administration category, the translation of the instructions of LSAS-SR was carried out together with the translation procedure of the questionnaire as described in the second category, the development of the questionnaire.

Fifth, in the scoring and interpretation category, this study included the analysis of gender and age-related to LSAS-SR.

Sixth, the documentation category was not included in this study.

2.1. Participants

A total of 2074 participants from the general population participated in this study. Their age ranged from 16 to 38 years old (M = 19.42, SD = 1.94). The age of the participants was in the range reported in the previous studies (16-87 years) [16, 34, 36, 40, 43]. 73% of the participants were female. It is similar to some studies that involved more female participants [16, 36, 40, 41, 43, 49], and different from some studies that involved more male participants [34, 42]. They lived in seven big cities in Indonesia, representing the major ethnic groups in Indonesia, i.e., Bandung (31%), Denpasar (16%), Banjarmasin (8%), Makassar (15%), Yogyakarta (10%), Padang (10%), and Medan (10%).

Participants of the current cross-sectional study were selected using multi-stage cluster sampling with the following steps. In the first step, the cluster was applied based on the geographic region in Indonesia, namely the Java region, Sumatra region, Kalimantan region, Sulawesi region, Bali, East Nusa Tenggara, Maluku, and Irian region. In the second step, the cluster was applied to select the city/cities in every geographical region. This second step was carried out based on the information about the main ethnic groups, which were the majority of the population in each region. Seven big cities were selected in this second step, namely Bandung and Yogyakarta (Java region), Medan and Padang (Sumatra region), Banjarmasin (Kalimantan region), Makassar (Sulawesi region), and Denpasar (Bali, East Nusa Tenggara, Maluku, and Irian region). In the third step, the cluster was applied to select the universities/schools in each selected city. This third step was followed by the fourth step to determine the unit of analysis.

In the current study, the authors recruited an instructor in every city. They were recruited from personal contacts of the authors who met the following criteria, namely psychologists who were willing to participate in the current study as an instructor and were willing to do the tasks according to their responsibilities. These instructors were responsible for taking care of permission from the institutions (e.g., university) where the potential participants studied or worked, inviting the potential participants, managing data collection, including scheduling and conducting data collection, as well as obtaining informed consent from the participants. Participants who studied or worked at the institution were expected to give access to the instructor to get participants outside the institution. This technique was applied particularly to get the participants in their senior age. The invited potential participants should meet the following criteria, namely, aged 16 years and over, male or female, not a person with a mental disorder, and willing to give their consent. The potential participants who were interested in participating in the study contacted the instructor, and the instructor put their names on the list of participants. The authors organized a virtual workshop for the instructors for the data collection procedure. The instructor's recruitment was conducted in May 2019, and the data collection was conducted from August 15th 2019, to November 15th 2019.

The total amount of participants obtained in this study was in accordance with [56]. The total amount of participants also exceeded the number of participants in previous studies [37, 40, 44, 45, 49, 51].

The participants received the information a day before data collection, and their written consents were collected before the session began. Their participation was completely voluntary, and they received a participation reward at the end of the session. The participants could withdraw their participation at any time during the period of data collection. They could also be absent for various reasons, for example, being sick or having class.

2.2. Instruments

This study administered paper-and-pencil self-report instruments. They were the Liebowitz Social Anxiety Scale – Self Report (LSAS-SR) [21], the Brief Fear of Negative Evaluation Scale (BFNE) [20], and the Taijin Kyofusho Scale (TKS) [57]. The BFNE and TKS were used in the current study to examine the convergent validity of LSAS-SR-Indonesia.

Table 1. Descriptive statistics and coefficient of Cronbach’s Alpha.
Variables Items Min. Max. Mean SD α
LSAS total score 48 3 126 50.94   20.13 0.92
LF total score 24 0 62 25.04   11.27 0.88
LA total score 24 1 64 25.90   10.38 0.84
LFP 13 0 33 12.81   5.96 0.76
LAP 13 0 33 12.93   5.59 0.70
LFSI 11 0 32 12.23   6.10 0.82
LASI 11 0 32 12.97   5.70 0.78
BFNE 12 11 45 29.06 5.83 0.75
TKS 31 31 205 118.70 31.74 0.93
Notes: LF = LSAS Fear, LA = LSAS Avoidance, LFP = LSAS Fear Performance, LAP = LSAS Avoidance Performance, LFSI = LSAS Fear Social Interaction, LASI = LSAS Avoidance Social Interaction, α=coefficient of Cronbach’s Alpha. Source: Primary data.

2.3. The Liebowitz Social Anxiety Scale (LSAS) [21]

The LSAS, as previously mentioned, consisted of 24 items presenting social and performance situations. The fear and avoidance of the participants in every situation were measured [21]. This study used LSAS-SR. The LSAS consisted of 24 items, of which 11 items were for social interaction situations and 13 items were for performance situations. Each situation had two 4-point Likert scales, one for the intensity of fear (0 = None, 1 = Mild, 2 = Moderate, 3 = Severe) and the other for the frequency of avoidance of the situation (0 = Never, 1 = Occasionally, 2 = Often, 3 = Usually). Its further description, including the psychometric properties of the scale, was presented in the introduction. The Cronbach's alpha coefficients of the Indonesian version of the scale (LSAS-SR-Indonesia) in the current study were 0.92 for LSAS-total, 0.88 for LSAS Fear, and 0.84 for LSAS Avoidance (Table 1).

2.4. The Brief Fear of Negative Evaluation Scale (BFNE) [20]

A brief version of the FNE [20] is a self-report questionnaire that assesses participants’ fear related to a negative evaluation from others. The BFNE consisted of 12 items. The current study used the Indonesian version of BFNE with a 4-point Likert scale. The participants were requested to respond to each item ranging from 1 (Strongly Unfavorable) to 4 (Strongly Favorable). Some studies showed good internal consistency of the scale, such as 0.90 [20], 0.86 [58], and 0.91 [59]. The Cronbach's alpha coefficient of the Indonesian version of the scale in the current study was 0.75 (Table 1), indicating that the internal consistency of the scale was good. FNE was a feature of social anxiety and was proposed as the core concept of social anxiety, e.g., [11, 20, 60, 61]. Previous studies demonstrated that fear of negative evaluation had a positive correlation with social anxiety [17, 62-64]. Based on these results and its psychometric properties, the current study used this scale for examining the convergent analysis of LSAS-SR-Indonesia.

2.5. The Taijin Kyofusho Scale (TKS)

The Taijin Kyofusho Scale (TKS) [57] is a self-report questionnaire that assesses the symptoms of taijin kyofusho. Taijin kyofusho is a fear of giving offense to others persistently and excessively in social circumstances by physical characteristics such as gaze, blushing, or emitting foul body odor [57, 65]. In DSM-V [1], it is classified under social anxiety disorder (SAD), briefly described under culture-related diagnostic issues, and in ICD-10 [66], it is known as anthropophobia under social phobia. Recent studies demonstrated that taijin kyofusho is no longer described as a unique culture-bound form of social anxiety [67-69] as previously introduced [41, 70]. TKS consisted of 31 items. The concerns about doing something to offend or embarrass others were reflected on a 7-point Likert scale, ranging from 1 (totally false) to 7 (exactly true). The English version of the scale demonstrated a good internal consistency with the Cronbach's alpha coefficient of 0.93 [57]. In the current study, the Indonesian version of TKS was used to assess the taijin kyofusho symptoms of the participants. The Cronbach's alpha coefficient of the Indonesian version of the scale in the current study was 0.93 (Table 1), showing good internal consistency of the scale. TKS had a positive correlation with social anxiety [16]. Based on these correlations and their psychometric properties, this study used this scale to assess the convergent analysis of LSAS-SR-Indonesia.

2.6. Procedures

This study was approved by the Research Ethics Committee of Universitas Padjadjaran, Bandung, Indonesia, with a reference number of 1055/UN6.KEP/EC/2019. This study received informed consent from all participants. All procedures in the study were in accordance with the ethical standards of the Institutional Research Ethics Committee. This study received permission from Mr. Michael R. Liebowitz to examine the psychometric properties of the Indonesian version of LSAS-SR (LSAS-SR-Indonesia).

The data were collected in a group of 20 participants in a classroom following the data collection procedure. The instructors encouraged the participants to fill out the questionnaires. They completed the questionnaires in the classroom with a pencil, and they were allowed to keep the pencil as a participation reward. As recommended by the Research Ethics Committee, the participants received a box containing snacks and a glass of mineral water. The completion of the questionnaires, including the instructions and the collection of their consent, took approximately 50 minutes.

The data collection was carried out in seven cities, namely Bandung, Denpasar, Banjarmasin, Makassar, Yogyakarta, Padang, and Medan. The authors organized monitoring of the data collection twice a week using virtual meetings. The instructors reported the course of data collection twice a week via email. It was reported that the data collection was carried out according to the data collection procedure.

2.7. Data Analysis Techniques

Data input was carried out according to the coding system of each instrument. Data analysis was then performed. The data were excluded from the analysis if 25% or more of the items were missing. If the missing items were less than 25%, the data were estimated by means of multiple imputations using regression as implemented by SPSS v.21. In the current study, the missing data was found in Yogyakarta (3 missing data). The statistical analysis in this study used two computer programs, i.e., SPSS v.21 and LISREL v.10.3, to examine descriptive analysis, internal consistency, item discrimination, and validity analysis of LSAS-SR-Indonesia including convergent validity, internal structure validity, factorial validity, and content validity.

The internal consistency of LSAS-SR-Indonesia was analyzed using the coefficient of Cronbach’s alpha. The item discrimination was examined by calculating the correlation of the corrected item-total. The convergent validity of LSAS-SR-Indonesia was conducted by analyzing the correlation with the Brief Fear of Negative Evaluation Scale (BFNE), and the Taijin Kyofusho Scale (TKS) using Pearson Product Moment Correlation. The internal structure validity of LSAS-SR-Indonesia was conducted using Pearson Product Moment Correlation between LSAS subscales (LSAS Fear and LSAS Avoidance) and their subscales (LSAS Fear Performance, LSAS Avoidance Performance, LSAS Fear Social Interaction, LSAS Avoidance Social Interaction). Factorial validity analysis was conducted to analyze the construct validity of LSAS-SR-Indonesia. This study examines four-factor models: 1) two models for single-factor model (i.e., model 1, a single-factor model for fear; and model 3, a single-factor model for avoidance), and 2) two models for two-factor model (model 2, two factor model for fear; and model 4, two-factor model for avoidance). These models were described as follows:

Model 1: One factor for Fear scale, consists of 24 item loads in the factor Fear.

Model 2: Two-factor for Fear scale, consists of 13 item loads in the factor Fear Performance and 11 item loads in the factor Fear Social Interaction.

Model 3: One factor for Avoidance scale, consists of 24 item loads in the factor Avoidance.

Model 4: Two-factor for Avoidance scale, consists of 13 item loads in the factor Avoidance Performance and 11 item loads in the factor Avoidance Social Interaction.

Factorial validity was analyzed using Confirmatory Factor Analysis (CFA). The model of CFA was analyzed using the goodness of fit criteria, which were root mean squared error of approximation (RMSEA), root mean squares residual (RMR), non-normed fit index (NNFI), comparative fit index (CFI), and adjusted goodness-of-fit index (AGFI). Smaller values of RMSEA and RMR indicates a better fit, a value <0.10 indicates a good fit, and a value <0.05 indicates a very good fit. The values of NNFI, CFI, and AGFI ranged from 0 to 1.0, and value >0.9 indicated a good fit to the data [71].

The content validity of LSAS-SR-Indonesia was carried out using the evaluation of an expert panel. The analysis of gender and age was conducted using a 2 x 2 factorial analysis of variance (ANOVA). The two-way ANOVA was conducted to examine the difference in gender, age, and their interaction with the fear and avoidance subscales.

3. RESULTS

3.1. Descriptive Analysis

Results in Table 1 show that the LSAS total score of the participants (M = 50.94) was generally lower than the score in the study of [40] and higher than [36, 45, 51]. In this current study, the fear score (LF, M = 25.04) was lower than the score in the study [40] and higher than [36, 45, 51]. The avoidance tendency in facing the perceived anxious social situation (LA, M = 25.90) was higher than the findings of the previous studies [36, 40, 49] and lower than [51]. In this current study, the tendency of participants to avoid social situations had a higher score (LA, M = 25.90) than the tendency to fear (LF, M = 25.04). This result was similar to the findings of [51], but contradicted several previous studies, in which the participants’ fear was higher than their tendency to avoid social situations [36, 40, 49].

3.2. Internal Consistency and Item Discrimination

The Cronbach’s alpha coefficients for LSAS-SR-Indonesia (see Table 1) were generally above 0.7 (LSAS total = 0.92, LSAS Fear = 0.88, LSAS Avoidance = 0.84, LSAS Fear Performance = 0.76, LSAS Avoidance Performance = 0.70, LSAS Fear Social Interaction = 0.82, LSAS Avoidance Social Interaction = 0.78). This result shows that LSAS-SR-Indonesia had good internal consistency. These results are similar to the results of several previous studies, namely [43, 51] for the total scale, [36, 37, 40, 43, 52], for the fear scale, and [36, 37, 40, 43, 51, 52] for avoidance scale. The study shows the measurement error estimation ranges from 8% to 30%.

The corrected item–total correlation is presented in Table 2. The majority of the items had good discrimination due to the absence of negative corrected item–total correlation. The corrected item–total correlation coefficient of the subscale, either in fear or in avoidance scale, was lower than the social interaction subscale. The majority of the scores were higher than 0.20, except for the one item of the avoidance scale that measured the subscale (Item 4). This result was similar to the study of [44].

Table 2. The corrected item–total correlation.
LSAS Items Corrected item – Total Correlation
LSAS total 48 0.25 – 0.57
LF total 24 0.25 – 0.60
LA total 24 0.26 – 0.52
LFP 13 0.22 – 0.52
LAP 13 0.20 – 0.50
LFSI 11 0.41 – 0.62
LASI 11 0.30 – 0.54
Notes: LF = LSAS Fear, LA = LSAS Avoidance, LFP = LSAS Fear Performance, LAP = LSAS Avoidance Performance, LFSI = LSAS Fear Social Interaction, LASI = LSAS Avoidance Social Interaction. Source: Primary data.

3.3. The Convergent Validity and the Internal Structure Validity

This study used Pearson Product Moment Correlation to examine validity. The first result is presented in Table 3, including the correlation between LSAS scores (LSAS total, LSAS Fear, LSAS Avoidance, LSAS Fear Performance, LSAS Fear Social Interaction, LSAS Avoidance Performance, and LSAS Avoidance Social Interaction) and BFNE and TKS scores. The result shows a significant correlation between LSAS (total and its subscales) and BFNE (Pearson’s r = 0.27 – 0.40, p< 0.001), which is similar to the previous study on convergent validity using FNE [40]. The result indicates a significant correlation between LSAS (total and its subscales) and TKS (Pearson’s r = 0.38 – 0.50, p< 0.001). These results proposed a good convergent validity of the total score and the two subscales of LSAS-SR-Indonesia.

Table 3. The correlation between LSAS with BFNE and TKS.
Variables   LFP    LAP    LFSI   LASI   LF total   LA total   LSAS Total
BFNE   0.37    0.27    0.37   0.29   0.40   0.30 0.38
TKS   0.47    0.40    0.45   0.38   0.50   0.43 0.49
Notes: All correlations were statistically significant at p<0.001. LF = LSAS Fear, LA = LSAS Avoidance, LFP = LSAS Fear Performance, LAP = LSAS Avoidance Performance, LFSI = LSAS Fear Social Interaction, LASI = LSAS Avoidance Social Interaction. Source: Primary data.

The second result is presented in Table 4 below, showing the correlation between the scores of LSAS Fear and LSAS Avoidance, and their subscales (Performance and Social Interaction). The result shows that all correlations were significant (p<0.001) at the level of 0.1%, ranging from 0.51 to 0.93. It is included the correlation between the scores of LSAS total and its subscales (LSAS Fear and LSAS Avoidance) with their subscales (LSAS Fear Performance, LSAS Avoidance Performance, LSAS Fear Social Interaction, LSAS Avoidance Social Interaction). The result shows that the construct of LSAS-SR-Indonesia was empirically supported. These findings are in line with the findings of [38].

3.4. The Factorial Validity

Table 5 shows a goodness-of-fit index for the fitted factor models regarding the factorial validity of the four-factor models of LSAS-SR-Indonesia.

The analysis of Goodness-of-Fit shows that all four-factor models fit the data, with RMSEA, RMR, NNFI, CFI, and AGFI falling in the fit range. The result in Figs. (1 and 2) showed that all items in all four-factor models had a significant loading factor at the level of 5%. Model 1 had loading factors of fear ranging from 0.27 to 0.58. Model 2 had loading factors of the fear performance ranging from 0.27 to 0.56, and the fear of social interaction ranging from 0.45 to 0.59. Model 3 had loading factors of avoidance ranging from 0.26 to 0.56. Model 4 had loading factors of the avoidance performance ranging from 0.26 to 0.57, and of the avoidance social interaction ranging from 0.45 to 0.59. The correlations between fear and avoidance scales as a factor were significant in model 2 (0.98) and model 4 (0.94). However, in general, the loading factors were lower compared to the other adapted versions, namely the Portuguese version [44], Spanish version [36], and French version [40]. These results indicated that the construct validity of LSAS-SR-Indonesia was empirically proven.

Table 4. The correlation between LSAS Total and LSAS Fear, LSAS Performance.
LSAS LFP LAP     LFSI   LASI   LF total LA total LSAS Total
LFP 1 0.69     0.75   0.55   0.93 0.68 0.87
LAP - 1     0.54   0.69   0.66 0.92 0.84
LFSI - -     1   0.72   0.94 0.68 0.88
LASI - - -   1   0.68 0.92 0.86
LF total - - - -   1 0.73 0.94
LA total - - - - - 1 0.92
LSAS Total - - - - - - 1
Notes: All correlations are statistically significant at p<0.001. LF=LSAS Fear, LA=LSAS Avoidance, LFP=LSAS Fear Performance, LAP=LSAS Avoidance Performance, LFSI=LSAS Fear Social Interaction, LASI=LSAS Avoidance Social Interaction. Source: Primary data.

Table 5. The factor analysis of Goodness-of-Fit statistics for confirmatory factor models.
- - Fear Subscale Avoidance Subscale
Model
- - 1 2 3 4
RMSEA < 0.08         0.03        0.03 0.03     0.03
RMR < 0.08         0.02        0.03 0.02     0.03
NNFI > 0.90         0.96        0.96 0.95     0.94
CFI > 0.90         0.97        0.97 0.96     0.96
AGFI > 0.90         0.97        0.96 0.96     0.96
Notes: RMSEA = Root Mean Square Error of Approximation, RMR = Root Mean Square Residual, CFI = Comparative Fit Index, AGFI = Adjusted Goodness-of-fit Statistic. Source: Primary data.

3.5. The Content Validity

The content validity of LSAS-SR-Indonesia was performed after the evaluation carried out by an expert panel revealing a high degree of relevance and representativeness of the LSAS-SR-Indonesia, with an agreement of 60–100%. Since previous studies did not report the content validity of the LSAS-SR, the content validity of LSAS-SR-Indonesia contributes to the psychometric studies of LSAS-SR.

There was a concern over Item 4 (Drinking with others in a public place, subscale Performance), due to its lowest loading factor (0.27 in model 1, model 2, and model 4; 0.26 in model 3). Drinking activity in public places did not culturally exist in most areas of Indonesia. According to the expert panel concerning the low loading factor of item 4, that might be one of the explanations.

Fig. (1). CFA’s loading factors of Model 1 and Model 2.
Source: Primary data.

Fig. (2). CFA’s loading factors of Model 3 and Model 4.
Source: Primary data.

Table 6. The mean and standard deviation of LSAS across gender and age.
Gender Subscale Age Total
< 19 Mean (SD) ≥ 19 Mean (SD)
Male Fear 20.75 (10.52) 20.74 (10.71) 20.74 (10.63)
Female Fear 20.74 (10.63) 26.52 (11.26) 26.64 (11.09)
Total Fear 25.09 (11.04) 25.04 (11.40) 25.06 (11.27)
Male Avoidance 25.09 (11.04) 25.04 (11.40) 25.06 (11.27)
Female Avoidance 27.33 (10.18) 26.59 (10.42) 25.84 (10.34)
Total Avoidance 26.20 (10.26) 25.75 (10.44) 25.91 (10.38)
Note: SD = Standard Deviation. Source: Primary data.
Table 7. Two factorial ANOVA.
LSAS Subscales Demography Variables F p-value η2
Fear Gender 110.70 0.00 0.05
Age 0.10 0.75 0.00
Gender*age 0.09 0.77 0.00
Avoidance Gender 45.67 0.00 0.02
Age 0.65 0.42 0.00
Gender*age 0.37 0.54 0.00
Note: Source: Primary data.

3.6. The Analysis of Gender and Age

This study used a 2 x 2 factorial analysis of variance (ANOVA) to analyze gender and age. Table 6 presented the descriptive analysis of LSAS-SR-Indonesia across gender and age, and Table 7 displayed the result of a two-way ANOVA analysis to examine if there were significant differences on gender and age in every subscale, i.e., fear and avoidance.

The result in Table 6 demonstrated that, in general, women scored higher than men in both fear and avoidance subscales (total scores). The scores in the current study were generally higher than the findings of [41]. The result in Table 7 showed that there were significant differences in subscale fear across genders, with F(1, 2063) = 110.70 and the effect size η2 = 0.05; as well as for the subscale avoidance, with F(1, 2063) = 45.67 and the effect size η2 = 0.02. These results were in line with the findings from [41]. There were no significant differences across age, contradicting the result found in [41]. There were no significant differences in the interaction between gender and age, both in fear and avoidance subscales. Since the previous study did not report the interaction between gender and age, both in fear and avoidance subscales related to the LSAS-SR [41], this analysis contributes to the psychometric studies of LSAS-SR.

LSAS-SR and its translation to the Indonesian version (LSAS-SR-Indonesia) are presented in the Appendix.

4. DISCUSSION

The current study examined the Indonesian version of LSAS-SR (LSAS-SR-Indonesia). The result showed that this version had good psychometric properties. The LSAS-SR-Indonesia had good internal consistency, and all items had good discrimination. It had a good convergent validity for the total score and the two subscales, and its construct was empirically supported. All four-factor models were fit to the data, and RMSEA, RMR, NNFI, CFI, and AGFI fell in the fit range. The loading factors obtained in the current study were significant. However, in general, the loading factors were lower as compared to the other adapted versions: Portuguese version [44], Spanish version [36], and French version [40]. These results and those obtained in consultation with the expert review panel revealed that LSAS-SR-Indonesia was an easy-to-use scale for the general population in Indonesia. This instrument could be used to assess social anxiety symptoms in Indonesia. This study discussed further analysis related to gender and age. The result showed that there were significant differences in fear and avoidance across gender, however, there was no significant difference in fear and avoidance across ages.

The current study has several strengths. The entire process followed the psychometric standard methods according to [53], including pre-condition category, test development category, confirmation category, administration category, as well as scoring and interpretation category. The current study received permission from Mr. Michael R. Liebowitz to examine the LSAS-SR-Indonesia. In the test development category, the adaptation process of LSAS-SR-Indonesia followed the standard procedure, including forward and backward translation, as well as the reconciliation process during the review panel that involved six experts. In the confirmation category, the current study included internal consistency and item discrimination, convergent validity and internal structure, factorial validity, and content validity. Using multi-stage cluster sampling, the current study involved a relatively large number of samples from seven big cities in Indonesia, representing the ethnic groups, respectively. This study provided evidence that LSAS-SR-Indonesia is a reliable and valid instrument for social anxiety symptoms in Indonesia.

However, the current study has several limitations. First, the test-retest method was not conducted in this study to examine the reliability, because the permission obtained from the institutions was only limited to one assessment. Second, this study only included participants from the general population with a mean age of 19.42 years, and the majority of them were female. Although multi-stage cluster sampling was applied in the current study, this study did not control the variability of the participants’ age and the proportion of the participants’ gender. Third, the current study was cross-sectional that could not explore sensitivity to change and predictive validity.

Future studies should consider a greater number of male participants, the age variability of the participants, and should include clinical samples as well as their comorbidity with depression, alcoholism, and quality of life. A longitudinal study is recommended for future studies to examine the sensitivity to change and predictive validity. If future clinical trials will be conducted to examine the effectiveness of the intervention, then sensitivity to change is specifically important to examine. Additionally, the development of a cut-off value for LSAS-SR-Indonesia is suggested. Following ITC Guidelines for Translating and Adapting Tests [53] in the sixth category, namely the documentation category, a manual containing a resume of future results’ studies should be provided.

CONCLUSION

This study proved that LSAS-SR-Indonesia was a reliable and valid instrument to assess social anxiety symptoms in Indonesia. It can be used to identify and assist in preparation for promotion and intervention programs in order to prevent the development of the symptoms.

LIST OF ABBREVIATIONS

LSAS = Liebowitz Social Anxiety Scale
BFNE = Brief Fear of Negative Evaluation Scale
TKS = Taijin Kyofusho Scale
CFA = Confirmatory Factor Analysis

ETHICS APPROVAL AND CONSENT TO PARTICIPATE

This study was approved by the Research Ethics Committee of Universitas Padjadjaran, Bandung, Indonesia, no. 1055/UN6.KEP/EC/2019. This study received informed consent from all participants.

HUMAN AND ANIMAL RIGHTS

No animals were used in this study. All human research procedures followed were, in accordance with the guidelines of the Helsinki Declaration of 1975.

CONSENT FOR PUBLICATION

Informed consent was obtained from all individual participants involved in this study.

STANDARDS OF REPORTING

STROBE guidelines were followed.

AVAILABILITY OF DATA AND MATERIALS

The authors confirm that the data supporting the findings of this research are available within the article.

FUNDING

This research was supported by the Ministry of Research, Technology and Higher Education of the Republic of Indonesia, for research funding. This funding source had no role in the process of this study, namely the concept development, study design, data collection, data analysis, interpretation of the data and article preparation. The role of the funder was limited to research funding.

CONFLICT OF INTEREST

The authors declare no conflict of interest, financial or otherwise, in this study.

ACKNOWLEDGEMENTS

The authors wish to thank the Ministry of Research, Technology and Higher Education of the Republic of Indonesia, for the research funding. We express our gratitude to Mr. Michael R. Liebowitz for the permission to examine the psychometric properties of LSAS-SR-Indonesia. We would like to thank all participants involved in this study for their participation and cooperation. We acknowledge the hard work and dedication of the instructors.

Appendix. LSAS-SR and LSAS-SR-Indonesia.
S.No. LSAS-SR [21] LSAS-SR-Indonesia
1 Telephoning in public (p) Menelepon di tempat umum (p)
2 Participating in small groups (p) Berpartisipasi dalam suatu kegiatan kelompok kecil (p)
3 Eating in public places (p) Makan di tempat umum (p)
4 Drinking with others in public places (p) Minum-minum bersama orang lain di tempat umum (p)
5 Talking to people in authority (s) Berbicara dengan orang yang memiliki wewenang (s)
6 Acting, performing or giving a talk in front of an audience (p) Berakting, pentas, atau berbicara di hadapan banyak ‘penonton’ (p)
7 Going to a party (s) Pergi ke pesta (s)
8 Working while being observed (p) Bekerja sambil diamati (p)
9 Writing while being observed (p) Menulis sambil diamati (p)
10 Calling someone you don’t know very well (s) Menelpon orang yang tidak terlalu Anda kenal (s)
11 Talking with people you don’t know very well (s) Berbicara tatap muka dengan orang yang tidak terlalu Anda kenal (s)
12 Meeting strangers (s) Bertemu dengan orang yang tidak terlalu Anda kenal (s)
13 Urinating in a public bathroom (p) Buang air kecil di toilet umum (p)
14 Entering a room when others are already seated (p) Memasuki ruangan ketika orang-orang sudah duduk di tempatnya (p)
15 Being the centre of attention (s) Menjadi pusat perhatian (s)
16 Speaking up at a meeting (p) Menyatakan pendapat dalam rapat (p)
17 Taking a test (p) Menjalani tes mengenai kemampuan, keterampilan atau pengetahuan Anda (p)
18 Expressing a disagreement or disapproval to people you don’t know very well (s) Mengungkapkan ketidaksepahaman atau celaan terhadap seseorang yang tidak begitu Anda kenal (s)
19 Looking at people you don’t know very well in the eyes (s) Menatap mata seseorang yang tidak begitu Anda kenal (s)
20 Giving a report to a group (p) Menyampaikan pidato yang sudah dipersiapkan kepada suatu kelompok (p)
21 Trying to pick up someone (p) Mencoba berkenalan dengan seseorang dengan tujuan percintaan atau hubungan seksual (p)
22 Returning goods to a store (s) Mengembalikan barang ke toko untuk mendapatkan pengembalian uang (s)
23 Giving a party (s) Mengadakan pesta (s)
24 Resisting a high pressure sales person (s) Menolak seorang penjual yang gigih (s)
Notes: s = social interaction situation; p = performance situation

REFERENCES

[1] American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders 5th ed. 2013.
[2] Stein MB, Stein DJ. Social anxiety disorder. Lancet 2008; 371(9618): 1115.
[3] Bystritsky A, Khalsa SS, Cameron ME, Schiffman J. Current diagnosis and treatment of anxiety disorders. P&T 2013; 38(1): 30-57.
[4] Biggs BK, Vernberg EM, Wu YP. Social anxiety and adolescents’ friendships: the role of social withdrawal. J Early Adolesc 2012; 32(6): 802-23.
[5] McNeil DW. Evolution of terminology and constructs in social anxiety and its disorders.Social anxiety: clinical, developmental, and social perspectives 2nd ed. 2010; 3-21.
[6] McNeil DW, Randall CL. Conceptualizing and describing social anxiety and its disorders.Social anxiety: clinical, developmental, and social perspectives 3rd ed. 2014; 3-26.
[7] National Collaborating Centre for Mental Health (UK). Social Anxiety Disorder: Recognition, Assessment and Treatment 2013. Available from: https://pubmed.ncbi.nlm.nih.gov/25577940/ [accessed: September 16, 2022]
[8] Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry 2005; 62(6): 593-602.
[9] Kessler RC, Petukhova M, Sampson NA, Zaslavsky AM, Wittchen HU. Twelve-month and lifetime prevalence and lifetime morbid risk of anxiety and mood disorders in the United States. Int J Methods Psychiatr Res 2012; 21(3): 169-84.
[10] Fehm L, Beesdo K, Jacobi F, Fiedler A. Social anxiety disorder above and below the diagnostic threshold: prevalence, comorbidity and impairment in the general population. Soc Psychiatry Psychiatr Epidemiol 2008; 43(4): 257-65.
[11] Rapee RM, Heimberg RG. A cognitive-behavioral model of anxiety in social phobia. Behav Res Ther 1997; 35(8): 741-56.
[12] Schneier FR, Heckelman LR, Garfinkel R, et al. Functional impairment in social phobia. J Clin Psychiatry 1994; 55(8): 322-31.
[13] Leichsenring F, Leweke F. Social anxiety disorder. N Engl J Med 2017; 376(23): 2255-64.
[14] Kessler RC, Stang P, Wittchen HU, Stein M, Walters EE. Lifetime co-morbidities between social phobia and mood disorders in the US National Comorbidity Survey. Psychol Med 1999; 29(3): 555-67.
[15] Safren SA, Heimberg RG, Horner KJ, Juster HR, Schneier FR, Liebowitz MR. Factor structure of social fears: The Liebowitz Social Anxiety Scale. J Anxiety Disord 1999; 13(3): 253-70.
[16] Vriends N, Pfaltz MC, Novianti P, Hadiyono J. Taijin kyofusho and social anxiety and their clinical relevance in indonesia and Switzerland. Front Psychol 2013; 4: 3.
[17] Mattick RP, Clarke JC. Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behav Res Ther 1998; 36(4): 455-70.
[18] Turner SM, Beidel DC, Dancu CV, Stanley MA. An empirically derived inventory to measure social fears and anxiety: the social phobia and anxiety inventory. J Consult Clin Psychol 1989; 1(1): 35-40. [http://dx.doi.org/10.1037/1040-3590.1.1.35].
[19] Watson D, Friend R. Measurement of social-evaluative anxiety. J Consult Clin Psychol 1969; 33(4): 448-57.
[20] Leary MR. A brief version of the fear of negative evaluation scale. Pers Soc Psychol Bull 1983; 9(3): 371-5.
[21] Liebowitz MR. Social Phobia. Mod Trends Pharmacopsychiatry 1987; 22: 141-73.
[22] Osório FL, Crippa JA, Loureiro SR. Instruments for the assessment of social anxiety disorder: Validation studies. World J Psychiatry 2012; 2(5): 83-5.
[23] Asnaani A, Rinck M, Becker E, Hofmann SG. The effects of approach-avoidance modification on social anxiety disorder: a pilot study. Cognit Ther Res 2014; 38(2): 226-38.
[24] Buckner JD, Heimberg RG, Schmidt NB. Social anxiety and marijuana-related problems: The role of social avoidance. Addict Behav 2011; 36(1-2): 129-32.
[25] Heuer K, Lange WG, Isaac L, Rinck M, Becker ES. Morphed emotional faces: Emotion detection and misinterpretation in social anxiety. J Behav Ther Exp Psychiatry 2010; 41(4): 418-25.
[26] Lange WG, Keijsers G, Becker ES, Rinck M. Social anxiety and evaluation of social crowds: Explicit and implicit measures. Behav Res Ther 2008; 46(8): 932-43.
[27] Mishra P, Kiran UV. Parenting style and social anxiety among adolescents. Internat J Appl Home Sci 2018; 5(1): 117-23. Available from: http://scientificresearchjournal.com/wp-content/uploads/2017/12/Home-Science-Vol-5_A-117-123-Full-Paper.pdf
[28] Parvez S. Parenting styles of socially anxious and non-anxious students. FWU J Soc Sci. 2013 Summer, FWU. J Soc Sci 2013; 7(1): 47-51.
[29] Takada K, Takahashi K, Hirao K. Measurement error in the Liebowitz Social Anxiety Scale: results from a general adult population in Japan. Int J Psychiatry Clin Pract 2018; 22(4): 289-95.
[30] Tauscher J, Kielbasa W, Iyengar S, et al. Development of the 2nd generation neurokinin-1 receptor antagonist LY686017 for social anxiety disorder. Eur Neuropsychopharmacol 2010; 20(2): 80-7.
[31] Heimberg RG, Horner KJ, Juster HR, et al. Psychometric properties of the Liebowitz Social Anxiety Scale. Psychol Med 1999; 29(1): 199-212.
[32] Cox BJ, Ross L, Swinson RP, Direnfeld DM. A comparison of social phobia outcome measures in cognitive-behavioral group therapy. Behav Modif 1998; 22(3): 285-97.
[33] Blanco C, Heimberg RG, Schneier FR, et al. A placebo-controlled trial of phenelzine, cognitive behavioral group therapy, and their combination for social anxiety disorder. Arch Gen Psychiatry 2010; 67(3): 286-95.
[34] Baker SL, Heinrichs N, Kim HJ, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: a preliminary psychometric analysis. Behav Res Ther 2002; 40(6): 701-15.
[35] Beard C, Rodriguez BF, Moitra E, et al. Psychometric properties of the Liebowitz Social Anxiety Scale (LSAS) in a longitudinal study of African Americans with anxiety disorders. J Anxiety Disord 2011; 25(5): 722-6.
[36] Caballo VE, Salazar IC, Arias V, Hofmann SG, Curtiss J. Psychometric properties of the Liebowitz Social Anxiety Scale in a large cross-cultural Spanish and Portuguese speaking sample. Br J Psychiatry 2019; 41(2): 122-30.
[37] Forni dos Santos L, Loureiro SR, Crippa JAS, Osório FL. Psychometric validation study of the liebowitz social anxiety scale - self-reported version for Brazilian Portuguese. PLoS One 2013; 8(7): e70235.
[38] Fresco DM, Coles ME, Heimberg RG, et al. The Liebowitz Social Anxiety Scale: a comparison of the psychometric properties of self-report and clinician-administered formats. Psychol Med 2001; 31(6): 1025-35.
[39] Hedman E, Ljótsson B, Rück C, et al. Internet administration of self-report measures commonly used in research on social anxiety disorder: A psychometric evaluation. Comput Human Behav 2010; 26(4): 736-40. [https://doi.org/10.1016/j.chb.2010.01.010].
[40] Heeren A, Maurage P, Rossignol M, et al. Self-report version of the Liebowitz Social Anxiety Scale: Psychometric properties of the French version. Canadian Journal of Behavioural Science / Revue canadienne des sciences du comportement 2012; 44(2): 99-107. [https://doi.org/10.1037/a0026249].
[41] Olivares J, Sánchez-García R, López-Pina JA. The Liebowitz Social Anxiety Scale for Children and Adolescents. Psicothema 2009; 21(3): 486-91. [PMID: 19622334].
[42] Schmits E, Heeren A, Quertemont E. The self-report Version of the LSAS-CA: Psychometric Properties of the French Version in a non-clinical adolescent sample. Psychol Belg 2014; 54(2): 181-98. [https://doi.org/10.5334/pb.al].
[43] Soykan Ç, Özgüven HD, Gençöz T. Liebowitz Social Anxiety Scale: The Turkish Version. Psychol Rep 2003; 93(3_suppl): 1059-69.
[44] Terra MB, Barros HMT, Stein AT, et al. Internal consistency and factor structure of the Portuguese version of the Liebowitz Social Anxiety Scale among alcoholic patients. Rev Bras Psiquiatr 2006; 28(4): 265-9.
[45] Nagata T, Nakajima T, Teo AR, Yamada H, Yoshimura C. Psychometric properties of the Japanese version of the Social Phobia Inventory. Psychiatry Clin Neurosci 2013; 67(3): 160-6.
[46] Oakman J, Van Ameringen M, Mancini C, Farvolden P. A confirmatory factor analysis of a self-report version of the Liebowitz Social Anxiety Scale. J Clin Psychol 2003; 59(1): 149-61.
[47] Sugawara N, Yasui-Furukori N, Kaneda A, et al. Factor structure of the Liebowitz Social Anxiety Scale in community-dwelling subjects in Japan. Psychiatry Clin Neurosci 2012; 66(6): 525-8.
[48] Dancey C, Reidy J. Statistics without maths for psychology 7th ed.. 2017.
[49] Yao SN, Note I, Fanget F, et al. L’anxiété sociale chez les phobiques sociaux: validation de l’échelle d’anxiété sociale de Liebowitz (version française). Encephale 1999; 25(5): 429-35.
[50] von Glischinski M, Willutzki U, Stangier U, et al. Liebowitz Social Anxiety Scale (LSAS): Optimal cut points for remission and response in a German sample. Clin Psychol Psychother 2018; 25(3): 465-73.
[51] Levin JB, Marom S, Gur S, Wechter D, Hermesh H. Psychometric properties and three proposed subscales of a self-report version of the Liebowitz Social Anxiety Scale translated into Hebrew. Depress Anxiety 2002; 16(4): 143-51.
[52] Bobes J, Badía X, Luque A, García M, González MP, Dal-Ré R. Validación de las versiones en español de los cuestionarios Liebowitz Social Anxiety Scale, Social Anxiety and Distress Scale y Sheehan Disability inventory para la evaluación de la fobia social. Med Clin (Barc) 1999; 112(14): 530-8.
[53] ITC Guidelines for Translating and Adapting Tests Int J Testing 2nd ed. 2018; 18(2): 101-34.
[54] Werner O, Campbell D. Translating, working through interpreters and the problem of decentering.Handbook of Cultural Anthropology New York: American Museum of National History 1970; 398-419.
[55] Polit DF, Beck CT. The content validity index: Are you sure you know what’s being reported? critique and recommendations. Res Nurs Health 2006; 29(5): 489-97.
[56] Wolf EJ, Harrington KM, Clark SL, Miller MW. Sample size requirements for Structural Equation Models: an evaluation of power, bias, and solution propriety. Educ Psychol Meas 2013; 73(6): 913-34.
[57] Kleinknecht RA, Dinnel DL, Kleinknecht EE, Hiruma N, Harada N. Cultural factors in social anxiety: A comparison of social phobia symptoms and Taijin Kyofusho. J Anxiety Disord 1997; 11(2): 157-77.
[58] Reichenberger J, Smyth JM, Blechert J. Fear of evaluation unpacked: day-to-day correlates of fear of negative and positive evaluation. Anxiety Stress Coping 2018; 31(2): 159-74.
[59] Carter MM, Sbrocco T, Riley S, Mitchell FE. Comparing fear of positive evaluation to fear of negative evaluation in predicting anxiety from a social challenge. J Exp Psychopathol 2012; 3(5): 782-93.
[60] Clark DM, Wells A. A cognitive model of social phobia.Social phobia: diagnosis, assessment, and treatment 1995; 69-93.
[61] Heimberg RG, Brozovich FA, Rapee RM. A cognitive behavioral model of social anxiety disorder: update and extension.Social anxiety: clinical, developmental, and social perspectives 2nd ed. 2010; 395-422.
[62] Heimberg RG, Hope DA, Rapee RM, Bruch MA. The validity of the social avoidance and distress scale and the fear of negative evaluation scale with social phobic patients. Behav Res Ther 1988; 26(5): 407-10.
[63] Heimberg RG, Mueller GP, Holt CS, Hope DA, Liebowitz MR. Assessment of anxiety in social interaction and being observed by others: The social interaction anxiety scale and the Social Phobia Scale. Behav Ther 1992; 23(1): 53-73. [https://doi.org/10.1016/S0005-7894(05)80308-9].
[64] Davidson JRT, Miner CM, De Veaugh-Geiss J, Tupler LA, Colket JT, Potts NLS. The Brief Social Phobia Scale: a psychometric evaluation. Psychol Med 1997; 27(1): 161-6.
[65] Takahashi T. Social phobia syndrome in Japan. Compr Psychiatry 1989; 30(1): 45-52.
[66] World Health Organization. International statistical classification of diseases and related health problems 10th revision 1992.
[67] Clarvit SR, Schneier FR, Liebowitz MR. The offensive subtype of Taijin-kyofu-sho in New York City: the phenomenology and treatment of a social anxiety disorder. J Clin Psychiatry 1996; 57(11): 523-7.
[68] Choy Y, Schneier FR, Heimberg RG, Oh KS, Liebowitz MR. Features of the offensive subtype ofTaijin-Kyofu-Sho in US and Korean patients with DSM-IV social anxiety disorder. Depress Anxiety 2008; 25(3): 230-40.
[69] Kim J, Rapee RM, Gaston JE. Symptoms of offensive type Taijin-Kyofusho among Australian social phobics. Depress Anxiety 2008; 25(7): 601-8.
[70] Kirmayer LJ. The place of culture in psychiatric nosology: Taijin kyofusho and DSM-III-R. J Nerv Ment Dis 1991; 179(1): 19-28.
[71] Kelloway EK. Using LISREL for structural equation modeling: a researcher’s guide 1998.