Monday, August 22, 2011

The Challenging Nature of Field Evaluations

This summer has been an interesting learning experience as my work has been focused on program evaluation. I've done evaluation work before, but mostly from a desk. In Iraq we had local monitoring and evaluation (M&E) staff who were responsible for site visits and data collection, and hired local consultants for larger-scale evaluations that included lots of surveys and focus group discussions. This time around I'm actually doing it myself - this is mostly nice, as you have control over the data being collected, but also showed me how much work field evaluations are, and how much I probably don't want to go further down the path of being an M&E expert!

Developing surveys and focus group questions in a new cultural context is tough - particularly if you don't speak the language. The resources at our disposal during the development of our evaluation tools was limited - not much support or review from USAID, or assistance from local staff. Having local staff tell you how your questions are likely to be translated and received is a big help. On the whole, I thought this went well, but sometimes found myself lamenting at the lack of opportunity to pilot our tools.

Let me also say that getting ALL THE WAY OUT to the communities we visited - poor rural farmers tend to live out in the middle of nowhere - was exhausting. We traveled 40-100 km down two-lane or gravel roads, over mountain passes with lots of hairpin turns and no guard rails, dodged giant trucks barreling down the opposite lane, and avoided hundreds and hundreds of pedestrians carrying firewood, sheep, goats, cattle, oxen and many, many miniature donkeys - not to mention the odd baboon, Ethiopian fox, hyena or hornbill.

Once you arrive, sit anywhere you can find (rickety chair, on a rock, on a step outside a crumbling building) and become the center of attention for a few hours. Talking with Ethiopians is a pleasant experience - they are direct and effusive and happy to share their stories with you. It's tough to watch an emaciated mother of five with a baby strapped to her back walk away from an interview in mud-covered plastic sandals and threadbare skirts, after having just told you how her family's monthly food supplies only last for a week. Solace comes in knowing that hopefully your presence in her village will yield some additional help for her and her family.

Ethiopians have been receiving humanitarian aid for decades. For many, our visit is not their first meeting with foreign aid workers and many of them are exceptionally savvy to the nature of the aid their communities receive. We've had very specific requests for microfinance projects or agricultural collectives from the beneficiaries we've met. When you enter a community skeptical about certain program elements and are repeatedly told that everything's generally being done as it should be, you wonder how accurate your results are - and whether you, as an outsider, are even capable of getting accurate information. At the end of the day, you trust the people you're speaking with and hope the stories they tell are true.

Now that we've completed our evaluation we will share our stories and go, and maybe hear something about the impact it's made on the program or future ones. The findings are interesting and hopefully not just relegated to email inboxes or computer file folders. I plan to follow the results closely in hopes of learning if our findings have made a difference in the aid that's provided to millions of hungry Ethiopians. I hope they do.

Thursday, August 4, 2011

So what are you doing over there, anyway?

Lest anyone think we are having a sightseeing and rock-climbing vacation over here, I thought I'd share a little bit about the work our team is doing in Ethiopia.

For the past several years, USAID has funded an emergency food assistance program in Ethiopia. Food security is a big issue here - there are millions of people in this country who have been identified as chronically food insecure - meaning they never have enough to eat. With 85% of the country's population of 80+ million people living in rural areas, most people rely on agricultural or pastoralist activities for subsistence and, if they're lucky, moderate income. People in the western part of the country enjoy greater food security due to more stable and productive crops, rainfall and natural resources. In the central and eastern parts of the country, however, the majority of the population outside of (and even within) major cities ekes out a living by whatever means they can - normally farming about one hectare of land and raising small herds of goats, sheep or cattle.

Farming is tough work here - people still use a pair of oxen and a crude wood and metal plow to till the earth. The land is unforgiving - in most parts, rocky and dry and relatively inhospitable. Ethiopia's topography does nothing to help - everywhere we visit is characterized by massive hillsides, challenging to farm and subject to major flooding and erosion. Every year, families struggle with the weather and the degraded quality of small tracts of land they don't even own. Overfarming, poor natural resource management, poor quality and lack of availability of seeds and other vital farming inputs mean that many of Ethiopia's rural farmers struggle to produce sufficient quantities of food to meet their family's food needs for the entire year. As the months from harvest time stretch on - and the shocks of flood or drought compromise yet another harvest - families find themselves without resources to provide for the most basic of needs.

Relying on the Government of Ethiopia's crop and livestock surveillance systems, USAID provides emergency food to acutely food insecure families - those who need some extra help for 3-9 months out of the year. A typical food ration per person per month includes 15 kilos of cereals (wheat or sorghum), 1.5 kilos of pulses (usually yellow split peas), and 0.5 liter of vegetable oil. 35% of beneficiaries also receive 4.5 kilos of corn-soy blend, a fortified food product directed at children under five and pregnant/lactating women, the most nutritionally vulnerable groups.

With this system in mind - providing assistance to more than 1.6 million beneficiaries per year - we are aiming to answer the following questions:
1. What is the system for targeting food aid beneficiaries? How are they selected? Are they getting enough food to meet their needs?
2. How are families and communities utilizing the food they receive? Are they sharing it? Selling it?
3. How are selected individuals targeted for corn-soy blend?
4. What do individuals and communities know about what corn-soy blend is, who it's for, and how it should be used? How are they using it? Are they sharing it? Selling it? Do they need more information about it?
5. Are people being required to work in exchange for the food rations they receive? What is the system?

To answer these questions, we're visiting 6 different areas throughout the country - two communities in each area. There, we conduct interviews and hold focus group discussions to learn about local attitudes and practices, as well as meeting with local government officials to get their impressions of the situation. We're meeting with about 120 people every week to talk about the above issues. At the end, we've been told by our USAID supervisor, we'll be "the US government's best informed people on food aid in Ethiopia."

We're finding a lot. Due to some sensitivities around food security issues in the country I'll wait to share those until I get back to the US. Ultimately, the food USAID distributes is doing a lot of good for a lot of people. Initiatives like the US government Feed The Future program - which focuses on increasing the capacity of local agricultural systems to provide local solutions to food insecurity - have incredible promise for long-term solutions to these problems. As we can see in the current crisis in this region, ensuring that local systems are in place to deal with natural disasters and shocks like droughts and floods, and to provide emergency food aid when needed, help to keep people alive. The Ethiopian Government has a pretty strong system in place to meet these needs when they arise - keeping them more resilient to the country's famines of the 1980s and 90s and the current crisis in Somalia, now bleeding over into Kenya and Ethiopia. They have done a lot to avoid these situations and learned from past tragedies. Hopefully the international community can help to strengthen these mechanisms in Ethiopia's neighbors.

Friday, July 29, 2011

Hike to the Hole-in-the-Wall Church

When the team and I first arrived in Mekele, capital city of Ethiopia's northern Tigray region, we had a day off. Lacking the inevitable travel fatigue this early into our journey, we decided to use our free time for some sightseeing. We stopped in at the tour office next to our hotel and were advised to take a trip to visit some of the region's rock-hewn churches. As Ethiopia is famous for its ancient Christian churches carved out of rock in myriad ways, we thought this sounded like a good cultural activity.

The highlight, said our guide before departure, would be the Abuna Yemata church, carved out of a cliff on the side of a giant monolith. Skeptically, we asked about the extent of the trek to the church. "Don't worry, anybody can do it," he said.

We drove out of the city in a minibus, down gravel roads winding through the highland plateaus of Tigray. The scenery was beautiful - huge rock ridges and sweeping valleys rich with dark soil and golden stone. This photo, taken from the road, is a typical view in the Tigrayan highlands:



We arrived at the site and met a few more guides ... guides-cum-militia, apparently, as one carried a dull, old axe for protection as he walked along with us, and another stood stationed outside our vehicle to make sure nothing was stolen or bothered. Good, sturdy Ethiopians, these two:



While they looked up for the climb, I'm not entirely sure the rest of us were. Having arrived a week or two prior in Ethiopia, our poor sea-level lungs were still not used to the elevation - 7000' or higher up in these highlands. It's a pretty serious change in altitude when you're not used to it - and that's just when you're standing still. Suffice it to say that looking at the following task in front of us was no joke:



Learning that we were climbing about halfway up the largest rock to reach the church (I know, "What church?!") didn't stop us. Our guides moved among the rocky terrain like mountain goats and we did our best to stumble along behind on the rough paths. As we climbed higher, we were treated to sweeping views of the valley - truly breathtaking (or was the breathtaking from the altitude and the hiking?).



After gaining about 1500 feet, we reached the end of the path - and looked up to a sheer, creviced rock wall. I wish I had thought to take out my camera - and I would've had I known what I was getting into. With the help of our nimble and experienced guides, up we went, hand over foot by the small holds in the rock, up one cliff and then another, until we were perched on the side of a giant rock monolith. Here is a break before the last scramble - you can just about see the church entrance up above the bushes:



After our final ascent, we tiptoed around a 2" wide ledge around the side of the cliff to reach the entrance to Abuna Yemata church - basically a small cave carved out of the side of a mountain 1500 feet in the air. The priest awaited us.




Inside the church we found original 4th century frescoes, telling the story of the original nine Syrian apostles and an Ethiopian prophet who delivered Christianity to Ethiopia. Persecuted by Muslim rulers in the 4th century, the first Ethiopian Christians carved hidden churches out of rock and mountain, creating a small and hidden faith that grew suddenly with the rise of the Axumite kings. The bible pictured below is written in the ancient Ge'ez language.




Despite the grueling and dangerous climb, the trip was fantastic. Between the views and the glimpse into Ethiopian history, it was one of the more unforgettable parts of my Ethiopia experience.

Thursday, July 28, 2011

No blogging in Ethiopia ... or is there?

"Hello after a long hiatus" seems to be a common theme of this blog, no? When I arrived in Ethiopia I found that Blogspot was blocked. The Ethiopian Government is, shall we say, quite involved in internet communications in country - so some websites are limited. However, after a month and a half, it now seems that I can access my blog - so here we go with another catch up!

I've been in country for six weeks now and it has been a great experience thus far. We spent our first week at the US Embassy learning about US Government operations here, particularly an overview of USAID but also information about the State Department and some other key agencies. I'm part of a group of 18 interns (17 still standing, with one gone home due to some combination of altitude sickness, heart problems and schistosomiasis), mostly grad students, mostly in international development programs but with other fields represented including public health, business, communications, public administration, etc. It's a great group and we're having a lot of fun.

I'm part of a team of four working on an evaluation of a USAID emergency food assistance program. Emily (masters in Foreign Service from Georgetown), Mary (masters in International Development from Johns Hopkins) and Rocco (masters in Public Administration from University of Vermont) are fantastic teammates - I'm lucky to be paired up with them and really enjoying having such a talented team.

Our first week also included development of an evaluation methodology and data collection tools (surveys, focus group discussion guides, etc.) to prep for our weeks of fieldwork. It also involved a crash course in the program we're evaluating, food aid in Ethiopia, and USAID's strategy for food assistance programming ... and meetings with five NGO partners who implement the program (CARE International, Save the Children UK, Catholic Relief Services, Food For The Hungry and the Relief Society of Tigray) to prepare for our visits to each organization's sites. Suffice it to say that we had very full brains after week one!

In late June we headed out for a month of fieldwork. We started in Ethiopia's Tigray region, in the north of the country. After a week's work around Mekele city, we headed south for a week in the town of Woldiya (a glorified truck stop) and continued on to the city of Lalibela, home of Ethiopia's famous rock-hewn churches. After an extra week in Lalibela to do some initial data management and report writing, we headed back to Addis Ababa for a week at the Embassy for briefings.

Saturday, we head off for another three weeks in the field, doing additional site visits and data collection in the Oromiya region, Dire Dawa municipality, and the Southern Nations, Nationalities & People's Region (SNNPR) in the south of the country.

It has been fantastic to travel throughout the country, meeting emergency food aid beneficiaries and learning about the way the program is helping them. Ethiopia has been well known for its food insecurity in past years, and the recurrence of bad weather and persistent spikes in global food prices continue to take their toll on poor farming communities. The food USAID is providing helps people to survive times of severe food shortage, and we are learning a great deal about how the program works and how beneficiaries are using the food they receive.

That's it as a brief overview - I'll do my best to play catch-up on the details of the past few weeks as long as I maintain blog access!

Tuesday, June 14, 2011

Leftovers of Insecurity

It was only my second day in Nigeria when I recognized a feeling long forgotten: the feeling of being unsafe. Walking down the aisles of a Lagos supermarket, despite the fact that I was surrounded by a swirl of ethnicities that included Nigerians, Chinese, South Africans, Indians and others, I couldn't shake the familiar feeling that I was under threat, being watched and needed to conclude my business and get to safety as quickly as possible. It was a clear carryover from Iraq, where short outings to the market or a local school truly were dangerous undertakings; I had to remind myself that while, as a large African mega-city Lagos is not without a need for street smarts, it is a free and safe society and I had nothing to worry about in a crowded shopping center.

The landscape here, in many ways, is not too different from Basra. The construction is the same both in style and its varying stages of completion; at first glance, haphazard, quick-fix methods with scarce resources seems to be the norm, and it does impact the outcome. Lagos has more greenery - slightly - but the dirt side roads and crush of vehicles feels familiar, as do the impromptu roadside stands selling whatever produce is in season, and the young men and boys rushing to car windows to sell bootleg DVDs, tissue boxes, fresh fruit or a window wash. In these instances, too, I have to remind myself that it is okay to be seen, that getting stuck in the traffic isn't putting me at risk, and that the only risk posed by roadside police stands is the demand for a bribe before you pass, and nothing more sinister.

The unwinding from a place like Iraq is a slow and subtle process; in some ways I have consciously felt it happen as habits disappear and walls begin to come down. In other instances it is striking when I realize it is taking place, as I see clearly what has evolved within me, and identify what I am letting go of. The conscious understanding of yet another way it has affected my thoughts and actions never fails to startle me, and while I know the process will continue, I also recognize there will be some ways in which it has changed me forever.

This is humbling in light of those who saw and suffered far more than I did. I experienced my share of risk and danger, was awakened many nights to alarms and explosions, and saw morale drop heavily among those around me in the wake of a soldier's or security contractor's death. For those who encountered harm and death head on, I wonder about their habits and walls compared to my own. No amount of training or strength of purpose prepares you for an experience like this, and I fear we are doing too little to care for those who have been deeply affected by this conflict, many of them just kids who seemed, in passing, so much younger and more immature than I. As I see and feel myself unwinding from my relatively easy experience, I wonder if they ever will.

Tuesday, June 7, 2011

Hello Again!

At long last, the blog has been resurrected. Thanks to all who followed my last journey - having friends and family along for the ride was great support and I appreciate all the calls for a return to blog posts! I hope to be more consistent on the current journey.

So, what's happened in the last year?

After a year and a half in Iraq, I left the program - and Mercy Corps - in November 2010. Oliver (the aforementioned boyfriend, who most of you have heard about at great length) and I spent six weeks traveling, with nearly a month in Thailand and Cambodia followed by spending the holidays in the UK with his family. It was great to check a few places off the must-visit list and dodge US taxes in the process!

I headed back to Portland in January to finish the last two terms of my masters coursework - which I completed last week! It's great to have the MPH [almost] completed - I will sit for my final exam in September.

In the meanwhile, I will be starting a 10-week internship with USAID, the US Agency for International Development, in Ethiopia in two weeks. I will be evaluating an emergency food distribution of a corn-soy blend food product, targeting pregnant/nursing women and children under five. I will be traveling to the cities of Mekele, Sekota, Lalibela and Dire Dawa, with an additional three weeks at the Embassy in Addis Ababa. It should be a great experience and I can't wait to get there.

Until then, I am hanging out in Lagos, Nigeria with Oliver, where he will be working until September on a technical/vocational training program. It's great to spend some time with him after nearly two months apart. (For those who were wondering, its now been a year and a half together - can't believe how quickly it's gone!) Lagos is a sprawling, overwhelming African mega-city - a stark contrast from the quiet of rural Ethiopia.

Many people have asked me if, knowing what I know now, I would take my job in Iraq again. My answer is overwhelmingly, yes. It was a fantastic learning experience - challenging, fascinating and career-changing. I've developed such confidence in my ability to travel, work and live abroad, to handle difficult and dangerous situations overseas, and to excel in development work. With conflict and insecurity under my belt, I have yet to hack it in the giant bugs/tents/pit latrines scenario - but I'm now more confident in my ability to navigate that with [relative] ease as well.

Thanks to all who are embarking on this new adventure with me - let the journey begin!

Fondly,
Alisha