Developer Research 101: The right methodology for reliable survey data

First, ask the right questions

Mind the source

It’s not only the size that matters

Is your data clean?

What is your margin of error?

Go for a large sample you can dig into

Are you lost?

  1. Ask the right questions. Make sure you accurately specify what business questions you need answered, and by which audience.
  2. Select the data collection method (such as a large-scale survey, telemetry, qualitative research, etc.) that is best suited for the problem you’re trying to solve.
  3. Carefully design your developer outreach to obtain a sample that is representative of the population you are interested in.
  4. Aim for a large sample, so that you may confidently dig into it, if you need to.
  5. Clean your data from illegitimate, or even fraudulent responses.
  6. If you’re confident enough that you have a random sample, estimate your margin of error — at the question, not survey, level.
  7. Check for sample bias and correct for any obvious deviations from randomness, without overfitting (or over-correcting).

--

--

--

We are the analysts of the developer economy. We help the world understand software developers - and vice versa! www.slashdata.co

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Functions in Python

Download tivimate apk Free

Download tivimate apk Free

Haskell Function, Type and Typeclass

CS 373 Fall 2020: Week of 2 Nov — 8 Nov

BENEFITS OF AGILE PROJECT MANAGEMENT

ALL ABOUT COUNTING SORT

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
SlashData

SlashData

We are the analysts of the developer economy. We help the world understand software developers - and vice versa! www.slashdata.co

More from Medium

Data Science Projects: Here’s How LONG they Take!

Your Bias Checklist!

Age of Data

Don’t let the CV algorithms beat you!