Evaluation leads to judgments about the worth of something (a product, an activity, a series of activities, an entire program). WordFarmers provides evaluation services as third-party evaluators: we are outside the project, even though we position ourselves at clients’ elbows.
- In our work, the judgments are usually something like this: “It looks like this activity went well, but it might be improved by changing this piece.” Such judgments might be delivered in conversation over a whole-day meeting, but we always produce written reports. This sort of evaluation is called “formative.” That means it helps ongoing work. This is the absolutely essential evaluation service.
- Sometimes, after a lot of formative work, WordFarmers helps projects prepare “summative” evaluation reports. These reports say why the project was worthy: it produced this much, products and services were relevant and useful, people learned what the project tried to teach. This is the least important part of what we do: but it’s still very important to clients and funders.
- Summative evaluation works for clients because formative evaluation helps do the heavy lifting as projects unfold.
Product development comes from a different part of R&D: the development part. This part is about creating or inventing something. Some thing. WordFarmers almost always develops these things in partnership with clients. What things?
- measurement instruments (surveys, questionnaires, interview questions)
- modules (instructional sequences posted on clients’ websites—including graphics, audio files, and video files)
- webinars (presentations with graphics, audio, and video elements)
- podcasts (audio products that may include scripts and material from interviews especially conducted for the podcast)
- books (we write copy to clients’ specifications, help clients consider graphic elements, and work with publishers to finalize the project)
- parts of any of these needed for client products.
Instructional design makes curriculum products accessible. It’s part of curriculum development and it requires a range of capacities for:
- video (producing, filming, editing)
- audio (producing, recording, editing)
- graphics (ideation, design, production)
- narrative (text calibrated for web platforms and learning management systems)
Research is a special kind of product development: studies. Research usually asks a big question to which the answer is unknown. For the clients of WordFarmers, such questions usually turn out to have implications for policy and practice. Some clients want answers so they can take part in policy discussions on the basis of evidence. Other clients want to test the effectiveness of educational practices or services. Legitimate studies begin with skepticism and proceed toward conclusions with many steps:
- literature review (What do we know? What don’t we know? Why?)
- conceptualization (How should we think about the issues? What ideas are relevant? What’s the research question?)
- methods (How can we answer the question? Do we need to ask real people? Which ones and in what way? How will we analyse the information we get back?)
- instrumentation (Creating valid and reliable instruments to use for collecting data: not a simple task)
- data collection and data analysis (Administering surveys, conducting interviews, managing the incoming data, and then using appropriate ways to condense the information received, aka data analysis)
- reporting and dissemination (Describing the study fully, presenting the findings clearly, and interpreting the conclusions for multiple audiences: WordFarmers prefers to see clients seek peer-reviewed publication and to produce short interpretive pieces for specialized audiences.)
Related activity means efforts not indicated above:
- proposal preparation (usually evaluation plans, but sometimes more than that)
- feasibility assessments
- informal literature or resource surveys
- short-term reviews of products or services to clients’ existing specifications
- consultations to discuss ideas or plans
- professional development (delivery of, from materials prepared with our help)