Methodology

This methodology provides detailed information on data collection, weighting, and imputation for the Small Business Credit Survey, and covers both employer and nonemployer data.

Data collection

The SBCS uses a convenience sample of establishments. A diverse set of partner organizations that serve the small business community contact businesses in their networks to invite participation in the survey.¹ The Federal Reserve Banks also directly contact prior SBCS participants and other small businesses from a variety of email lists.² The survey instrument is an online questionnaire that typically takes 8 to 12 minutes to complete, depending on the intensity of a firm’s search for financing. The questionnaire uses question branching and flows based on responses to survey questions. For example, financing applicants receive a different line of questioning than nonapplicants. Therefore, the number of observations for each question varies by how many firms receive and complete a particular question.

Weighting

Samples for the SBCS are not selected randomly; thus, the SBCS may be subject to biases that random sample surveys would not be subject to. For example, there are likely firms that are not on our contact lists, and this could lead to noncoverage bias. To control for some known potential biases, the sample data are weighted so that the weighted distribution of firms in the SBCS matches the distribution of the small firm population in the United States across a set of firm and owner characteristics.

Due to the structure and availability of US Census Bureau data used to compute the weights, employer firms and nonemployer firms are weighted in separate processes. Both employers and nonemployers are weighted by age, industry, geographic location (urban or rural location), gender of owner(s), and race or ethnicity of owner(s). Employer firms are additionally weighted by firm size (measured by number of employees) and census division.

We first limit the sample in each year to only employer or nonemployer firms. We then post-stratify respondents by their firm characteristics. Using a statistical technique known as “raking,” we compare the share of employer or nonemployer businesses in each category of each stratum (for example, within the industry stratum, the share of firms in the sample that are manufacturers) to the share of employer or nonemployer firms in the nation that are in that category. As a result, underrepresented firms in the convenience sample are up-weighted and overrepresented firms are down-weighted. We iterate this process several times for each stratum to derive a sample weight for each respondent. This weighting methodology was developed in collaboration with the National Opinion Research Center (NORC) at the University of Chicago. See the table below for sources of population distribution estimates used to construct the weights.

Race/ethnicity and gender imputation

Not every respondent provides complete information on the gender, race, and/or ethnicity of their business’s owner(s). We need this information to correct for differences between the sample and the population data. To avoid losing these observations, we use a series of statistical models to impute the missing data. Generally, when the models predict a given characteristic with an accuracy rate of around 80 percent, we use the predicted values from the models for the missing data. When the model’s accuracy rate is below that threshold, those data are not imputed, and the responses are dropped. After the models impute the data, we compare descriptive statistics of key survey questions with and without imputed data to ensure stability of estimates.

Comparisons across survey years

Changes to the questionnaire and weighting methodology limit over-time comparability on certain metrics. The time series data in the most recent SBCS reports supersede and are not necessarily comparable with the time series data in earlier publications. Please see the methodology sections from the annual releases of the Report on Employer Firms for a more complete explanation of the survey methodology and changes to the questionnaire by survey year.