From the Field From Silicon Valley to Tanzania: Putting AI to work for smallholder farmers

Journey into the field with Project Artemis, a new collaboration putting advanced phenotyping technology in the hands of breeders and smallholder farmers.

Our Land Cruiser plunges through thick dust, past goats picking at sparse weeds and Maasai herding cattle. The soil is cracked and dry, it has been over three months without rainfall.

Tourists know Arusha as Tanzania’s safari capital, a gateway to the Serengeti as well as the mountains Meru and Kilimanjaro. Our destination is in the shadow of Kilimanjaro; a patchwork of fields that farmers rely on for vegetables, fruit, and most importantly, beans – a valuable source of both dietary protein and income. But even with irrigation, the area is feeling the strain of climate change.

“This year because we didn’t have rains, the season has failed, the weather condition is very bad for every type of plant.”

Sembuli Mohamed Mkilaha on his farm near Chemka Village, Tanzania. He makes his livelihood by growing beans and maize, as well as raising cows. Credit: Georgina Smith

Parched farmland in the shadow of Mt. Kilimanjaro. Credit: Eliot Gee

Farmers are under increasing pressure, as weather becomes more extreme, pests and diseases more prevalent, and fertilizers more expensive (prices have more than doubled in the past year), to identify more resilient varieties of their staple crops. A bean variety that reaches maturity a few weeks earlier, is a bit hardier during a drought, tolerates pests and diseases, or produces a handful more pods per plant… this can be the difference between debt and hunger, or a well-fed and prosperous family. 

But how do farmers know which varieties are best suited to their fields? And how can crop breeders develop locally-adapted varieties to meet urgent, changing environmental conditions? It generally takes ten years to fully develop a new crop variety - a timescale that is no longer fast enough to keep up with climate change.

Better phenotyping = better climate adaptation

For thousands of years, farmers have observed their fields and used their senses to select the most promising varieties of crops; which plants have more pods, greener leaves, etc. In modern agriculture, we call this ‘phenotyping’, the core process driving crop varietal improvement. Phenotyping typically involves a clipboard, a pen, decades of experience, and a lot of patience. Besides being highly time-consuming, the process is subjective and prone to errors. For example, it is impossible to physically count all the open flowers in the field, so breeders use their best estimate. These estimates are not always accurate, since many breeding programs need to phenotype hundreds or thousands of plots, often in hot and humid conditions under the sun, straining the limited number of trained technicians.  

In recent years, digital imaging technology has created new possibilities for faster, more accurate measurement of plant characteristics. Digital phenotyping combines mass collection of photos from the field with rapid data analysis, powered by artificial intelligence (AI). Modern AI algorithms are able to quickly, efficiently, and accurately classify and count objects in an image.

Watch "Tech to Feed the Future". Video credit: Georgina Smith for the Alliance of Bioversity and CIAT 

For several years, the Alliance of Bioversity International and CIAT has been collaborating with Mineral, an Alphabet company applying the latest breakthroughs in machine learning, artificial intelligence, and perception technology to make agriculture more sustainable (Mineral recently graduated from X, Google’s research and development division or "moonshot factory" responsible for innovations like self-driving cars). Initial progress centered around a robotic rover outfitted with state-of-the-art circuitry and an arsenal of cameras that collect high-resolution images. With this data, AI is being steadily trained to phenotype plants, for example, to count every flower on every plant within a field throughout the entire growing season, a task that would be impossible to do manually. While these advanced technologies are being used by breeding programs in the Global North (the USA and Europe), these technologies are not accessible in the Global South; where agriculture, climate change, and poverty converge.

Smartphones are now ubiquitous across many areas in the Global South and have the potential to both source and provide useful agricultural information. Credit: Eliot Gee

A major factor limiting accessibility is cost.  High-tech imagery-based phenotyping tools can cost tens to hundreds of thousands of dollars, while smartphones are a small fraction of these costs. Therefore, Artemis is developing smartphone-deployable imaging technologies based on powerful AI that can be used in everyday phones, enabling breeders and farmers access to cutting-edge technology. But matching the technology with on-farm realities can be tricky as conditions in the Silicon Valley are completely different from realities in smallholder farms in the Global South.

Bringing AI to the field

This challenge has brought the Mineral team from California to rural Tanzania. They’ve arrived with questions: What are the current bottlenecks for plant breeders? How do farmers evaluate crop productivity? What are opportunities to apply technology for positive change in the current breeding system and farmers’ fields?

As we stand between the rows of bean plants, the local farmers share observations about everything from the number of pods on each plant to the number of leaves. For the technology engineers, this is the opportunity to consider the end user experience and anticipate how to overcome barriers for successful development and adoption.

Examining bean plants in the field. Agricultural production presents significant challenges for AI, for instance; multiple plant diseases and pests may be present in a single plant, flowers and pods may be hidden below leaves, flower and pod color can be variety dependent. Credit: Eliot Gee

Tally Portnoi is a MIT-trained AI engineer who recently joined Mineral. Her expertise in computer visioning focuses on training AI to derive useful information from images – originally for healthcare, and now for agriculture. She explains:

“I decided to transition to the agriculture sector to see how someone with my skills could make an impact on climate mitigation. It’s been extremely important to come here and speak directly to farmers [sic] and understand what factors into their decision making... For example, beans that lose their color or change color slower - why would we breed for something like this? Farmers care about this because it lets them hold on to the seed for longer, which gives them more control over getting a good price for their seed. I wouldn’t have an appreciation for this unless I came here and spoke with farmers directly.”

Next, the team heads to the Alliance’s Arusha field station, based at the Tanzania Agricultural Research Institute (TARI), to meet with Dr. Teshale Mamo. Teshale is a bean breeder by both profession and passion. Originally from Ethiopia, he studied agriculture in Italy and the USA before making the unconventional decision to return to fieldwork in Tanzania. He now counts decades of experience painstakingly developing locally-adapted varieties in many countries across the world.

Speaking with the Mineral team, Teshale explains that favorable characteristics are not just about field performance. He holds up his favorite variety, a round yellow bean, called ‘Njano’ in Swahili. The variety’s fast cooking time, good taste, and high market demand make it the preferred variety across northern Tanzania.

“Why do I have a passion for breeding? I want to change farmers’ lives and provide good, palatable varieties to the consumer. These days climate change is real, and in our breeding program we have been working to develop bean varieties that are resilient – climate-smart varieties. Our research focuses on consumer demand – the driver is the consumer, not us.”

Dr. Teshale Mamo and his bean plants in the Alliance’s field station greenhouse in Tanzania. Credit: Georgina Smith.

Taking a user-centered design approach, Artemis builds off the Mineral technology originally developed for the rovers currently at the Alliance’s hub in Colombia. The developers feel that the technology has matured enough for deployment as a mobile phone application that can widely collect and analyze images of crops. Given the ubiquity of smartphones across the world, lowering costs, and increasing reach in the Global South, this may be the transformative tool to bring plant breeding to the farmer – the ultimate goal of the Alliance and Mineral’s collaboration. Alliance scientist David Guerena, who leads Artemis, explains:

“There has historically been a disconnect between the formal plant breeding process and farmers. Breeding advancements are made by phenotyping, which requires controlled environments and is limited by the relatively few technically trained plant breeders. Artemis is a force multiplier, replicating the phenotyping process of plant breeders via the power of AI and ubiquity of smartphones. This technology holds promise to not only increase the accuracy of plant breeding, but bring the breeding process from the research station onto farmers’ fields.”

As the field visit ends, we enjoy a lunch of local beans, ‘tuko pamoja’, a Swahili phrase that means ‘we are together’. Researchers, breeders, engineers and farmers are together at the table, but also together to fight climate change, poverty, and food security through better crop breeding.

Alliance researchers, the Mineral team, and local farmers share lunch. Credit: Georgina Smith

About Artemis

The project is named after the ancient Greek goddess of the moon and nature. It is a “moonshot” project connecting mobile and AI technology with smallholder farmers and agricultural researchers, with the aim of creating a paradigm shift to the way we approach plant breeding. With the support of The Bill and Melinda Gates Foundation, and in collaboration with Mineral, the project aims to enable anyone, anywhere to collect accurate agricultural ground truth data to train and inform AI models to improve breeding, agronomic recommendations, and varietal placement.


Click here to learn more


For more information, email David Guerena: [email protected]