GUIDELINES AND PROCEDURES
This project surveyed universitity graduates. At the time of the survey, these students who had participated in the the labor market for were eligible for this type of study. Sampling method is the Proportional stratified random sampling method.
- First, each university member decides at least three majors included in the survey. After selecting the majors, the universities provide the lists of students who graduated in 2016 for the selected majors. Numbers of students in the lists that play as a survey population include student name, gender, place of birth, major, minor, type of training, email, telephone number,…
- Second, based on proportions of graduates in the each major, the numbers of students in the survey sample are determined. Survey sample are randomly selected by applying the Propionate Stratified Random Sampling Method. A statistic software is used for random selection. Data collection must be conducted in the selected sample.
- Third, replacement for the selected sample is also randomly prepared in advance. About 20% of the expected sample are reserved for replacement if the randomly selected graduates cannot be reached, or they refuse to participate the survey.
Sample size depends on the available financial capacity of university. It is NOT to send survey invitations to all graduates on the graduate list and to receive positive feedback from active graduated students. If so, the sample would be biased. Universities should comply with the above sample selection requirements.
The structure of the questionnaire is composed of five main sections: (1) Socio- demographic data; (2) Employment situation, (3) Employed graduates, (4) Unemployed graduates, (5) Inactive graduates. Each section includes multiple questions and information.
The socio-demographic section is composed by five items that ask subjects about:
- the University in which they obtained their (most recent) bachelor degree,
- the major of their bachelor degree,
- their gender,
- their age, and
- the year in which they initiated their bachelor’s degree
The second section is composed of two items that obtain information about:
- the number of jobs they had after graduation, and
- their employment situation at the time they answered the questionnaire.
This last question was used as a filter that allowed us to ask different questions to distinct groups of subjects depending on their employment situation (i.e., employed, self-employed, unemployed, inactive).
The third section is focused on employed graduates and is the longest section of the questionnaire. In this section, subjects initially provide information about their first job:
- Time to find first paid job after graduation
- Strategies to find first paid job after graduation
Then, they are asked to provide detailed information about their current job:
- Strategies to find current job after graduation
- Horizontal match
- Level of education required for current job
- Vertical Match
- Employer organization (type, sector, and size)
- Functional area
- Hierarchical level
- Formal written employment contract
- Part-time vs full-time
- Monthly income
- Work satisfaction
Importance of a number of competences for good performance
The fourth section is devoted to unemployed graduates. Here, subjects respond to questions about the following aspects:
- Time they have been looking for a job
- Job search strategies used
- Their difficulties to find a job
- Their opinion about the university career centers’ services
Finally, the fifth section is focused on inactive graduates. Here, subjects are asked about their reasons for not looking for a job.
Depending on the goals of survey, University can decide which should be asked.
Survey quality assurance
Quality of the survey is determined by many factors. Based on our research team’s experience, many activities/solutions for online surveys help to mitigate the quality gap between expectation and reality:
- The research team should apply the same sampling methods and data collection procedures as mentioned above;
- Anonymity of the survey was applied to mitigate the unbiased opinions and answers;
- Although we cannot do a random check for respondents with online surveys, other measures to ensure quality of the survey were applied. For example, one way is to put the time to complete the survey on a distribution curve, and surveys took a significant short time to complete were taken with care. Another way is to check and deal with surveys containing missing values.
Data collection procedures
An online survey is employed by using the SurveyMonkey, which is an online survey development cloud-based software as a service company. Respondents complete the survey questionnaire anonymously online. The survey procedure includes several steps.
Survey upload and pilot: The questionnaire was designed and uploaded online using the SurveyMonkey platform. This online survey was sent to each university to test whether any issues arise. Three graduates from each university participated in the pilot. One critical checking issue is smoothness, transitional questions, and shifts in the questionnaire. Some minor changes in questionnaires and shifts were appropriately made after the pilot to finalize online questionnaire.
Email confirmation: With telephone numbers provided in the lists, the selected graduates were called to confirm whether their active email was the one in the lists. If not, new email was asked to be provided. The confirmed emails were used to send out questionnaire to the selected graduates.
Participation invitation: An invitation letter, which was prepared in advance, was sent to the selected graduates via confirmed emails to ask for their participation. The SurveyMonkey generates unique links to the survey questionnaire for each participant, and the links were included in the invitation letter. The recipients just click “start survey” attached in the invitation letter to directly participate the survey.
Following-up: The selected graduates were friendly reminded three times during survey duration from December 2017 to March 2018. The reminder letter was also prepared in advance, and only graduates who had not completed the survey were reminded. After three tries at different time and graduates did not respond for the survey, a replacement was made by randomly drawn from the lists selected in advance.
Sampling bias checking
Sampling bias occurs when a sample statistic does not accurately reflect the true value of the parameters in the target population. In other words, the selected sample is practically impossible to ensure acceptable randomness and representation in sampling. Sampling bias is problematic because it is possible to give systematically erroneous inferences. Therefore, checking for sample bias is a necessary procedure in this survey.
First, the research team checks a sampling bias by using participation shares of each university. It is to check whether shares of survey participation by each university are significantly different from its shares in the total population. If the results do not statistically differ, the sample is unbiased. On the contrary, it is necessary to conduct additional surveys of selected people who have not yet answered.
Second, the research team checks a sampling bias by using shares of respondents by gender and majors. It is to check whether shares respondents in terms of gender and major in the sample are similar to their shares in the population. Empirical results show that there is little evidence of the sample bias using gender and major of graduates. If the results do not statistically differ among gender and majors, the sample is unbiased. On the contrary, it is necessary to conduct additional surveys of selected people who have not yet answered.