 |
 |
Planning Survey Design
The research objectives of the survey will
be formulated and the appropriate research design will be used
to address these objectives. For instance, if the purpose of the
study were to evaluate a new management procedure that is being
implemented, then an experimental research design would be the
most appropriate design. For this design, the survey questionnaire
would be administered prior to the implementation of the new management
procedure. The unit of observation (individual employees, departments,
divisions, etc) would be randomly assigned to either an experimental
or control group. After the new management procedure has been in
place within the experimental group for a period of time, a follow-up
survey questionnaire will be administered to both the control group
and the experimental group. Differences between the two groups
on the follow-up survey can be attributed to the new management
procedure.
Determining the Proper Survey Data Collection Methodology
Based
on the research objectives and agenda, we, in conjunction with
members of the Federal Government, will determine the appropriate
method of data collection for the study. These methodologies include
(but are not limited to) direct questionnaire administration, telephone
administration of surveys, mail-out questionnaires, web based surveys,
direct assessments, and observational methods.
Sampling
We will construct the sampling framework based on the
population (i.e. all employees within an agency, all households
in the United States) that is the focus on the survey. We will
prepare plans for cross-tabulation of the appropriate demographic
characteristics of the population such as age, education, family
income, gender, geographic area, household size, length of residence,
marital status, number of adults, number of children, political
party, religion, and race. We will randomly sample individuals
from this framework for inclusion in the study.
Survey Development
The research objectives will guide the development
of the survey instrument. If possible, we will recommend measures/questions
that are from established instruments or have been used in existing
surveys. Using items from existing surveys will provide the option
of benchmarking findings from the new survey against the existing
survey. For those objectives or issues where there are no existing
survey questions, we will design the appropriate measures.
Administering Surveys using various types of Data Collection Methods
We have the capabilities to administer surveys using various types
of data collection methodologies. For example, we have a call-center
capable of administering a large telephone based survey. We also
have the expertise to develop and administer a web-based survey.
Pretest/Pilot
Surveying
We will administer a pretest/pilot of the
survey or of individual items, as needed. A pretest or pilot survey
can be useful in determining the length of time that the survey
will require to administer. It can also be useful in establishing
the protocols and procedures necessary to administer the survey
efficiently. In addition, a pretest/pilot survey may allow us to
check the validity of the questions that are being asked in the
questionnaire.
Assessing Reliability and Validity of Data
We will assess the
reliability and validity of the survey data using multiple techniques.
The internal reliability of scales in the survey can be measured
using Cronbach's alpha. Cronbach's alpha measures how well each
item in the scale measures the intended construct. We will also
assess the construct and predictive validity of the questions or
scales in the survey. Construct validity is tested when the scale
or item that is being assessed is related another scale or item
that is tapping into the same construct, but is measured differently.
Predictive validity is met when the scale or item predicts other
constructs that should be related to the construct based on theory
or previous research.
Analyses of Quantitative and Qualitative Survey Data
Multiple
techniques will be used to analysis both quantitative and qualitative
survey data. Analyses of variance, Regression, and Chi-square distribution
are examples of statistical techniques that we have the capability
to perform. Our staff also has training in advanced statistical
techniques such as structural equation modeling, growth curve analyses,
event history analyses, and hierarchical linear modeling.
Production of Reports
We have experience in producing reports
for the Federal Government based on survey data. These reports
have included a description and summary of results, a description
of data collection and survey administration methods, and a discussion
of sample characteristics and the representative nature of data.
We have also worked with federal agencies to produce documentation
of survey instruments. The documentation has included section such
as, a description of the measures, reliability and validity assessments
that were performed, and analyses of non-response.
How Survey Services Support MOBIS
The Mitchell Group will provide
expert consultation, assistance, and deliverables associated with
all aspects of surveying within the context of MOBIS. Our personnel
will assist with, and/or perform all phases of the survey process
to include, but not limited to: planning survey design; sampling;
survey development; pretest/pilot surveying; assessing reliability
and validity; administering surveys using the various types of
data collection methods as appropriate (e.g., computer-assisted
surveying, focus groups, written questionnaires, in-person and
telephone interviewing); database administration; and analyses
of quantitative and qualitative survey data. Production of reports
to include, but not limited to: description and summary of results
with associated graphs, charts, and tables; description of data
collection and survey administration methods; discussion of sample
characteristics and representative of data; analysis of non-response;
and briefings of results to include discussion of recommendations
and follow-up actions. Surveys can provide:
- Needs assessments in education, leadership development, workforce
development, training and human resources
- Preparation of recommendations for basic and higher education;
institutional strengthening; human resource development; information
and communications technologies.
- Impact assessments of education sector programs on national
and regional development
- Technical, economic and social feasibility analyses
- Infrastructural needs assessment and capital development technical
assistance for national school systems.
Surveys can evaluate results of development activities, including
that of community and stakeholder participation. These evaluations
include program assessments and, when appropriate, recommendations
for the resdesign and improvement of programs.
|
