This summer has been an interesting learning experience as my work has been focused on program evaluation. I've done evaluation work before, but mostly from a desk. In Iraq we had local monitoring and evaluation (M&E) staff who were responsible for site visits and data collection, and hired local consultants for larger-scale evaluations that included lots of surveys and focus group discussions. This time around I'm actually doing it myself - this is mostly nice, as you have control over the data being collected, but also showed me how much work field evaluations are, and how much I probably don't want to go further down the path of being an M&E expert!
Developing surveys and focus group questions in a new cultural context is tough - particularly if you don't speak the language. The resources at our disposal during the development of our evaluation tools was limited - not much support or review from USAID, or assistance from local staff. Having local staff tell you how your questions are likely to be translated and received is a big help. On the whole, I thought this went well, but sometimes found myself lamenting at the lack of opportunity to pilot our tools.
Let me also say that getting ALL THE WAY OUT to the communities we visited - poor rural farmers tend to live out in the middle of nowhere - was exhausting. We traveled 40-100 km down two-lane or gravel roads, over mountain passes with lots of hairpin turns and no guard rails, dodged giant trucks barreling down the opposite lane, and avoided hundreds and hundreds of pedestrians carrying firewood, sheep, goats, cattle, oxen and many, many miniature donkeys - not to mention the odd baboon, Ethiopian fox, hyena or hornbill.
Once you arrive, sit anywhere you can find (rickety chair, on a rock, on a step outside a crumbling building) and become the center of attention for a few hours. Talking with Ethiopians is a pleasant experience - they are direct and effusive and happy to share their stories with you. It's tough to watch an emaciated mother of five with a baby strapped to her back walk away from an interview in mud-covered plastic sandals and threadbare skirts, after having just told you how her family's monthly food supplies only last for a week. Solace comes in knowing that hopefully your presence in her village will yield some additional help for her and her family.
Ethiopians have been receiving humanitarian aid for decades. For many, our visit is not their first meeting with foreign aid workers and many of them are exceptionally savvy to the nature of the aid their communities receive. We've had very specific requests for microfinance projects or agricultural collectives from the beneficiaries we've met. When you enter a community skeptical about certain program elements and are repeatedly told that everything's generally being done as it should be, you wonder how accurate your results are - and whether you, as an outsider, are even capable of getting accurate information. At the end of the day, you trust the people you're speaking with and hope the stories they tell are true.
Now that we've completed our evaluation we will share our stories and go, and maybe hear something about the impact it's made on the program or future ones. The findings are interesting and hopefully not just relegated to email inboxes or computer file folders. I plan to follow the results closely in hopes of learning if our findings have made a difference in the aid that's provided to millions of hungry Ethiopians. I hope they do.
No comments:
Post a Comment