MAE Research Project 2023-2025:
Planning and Reinforcement Learning for
Precision Agriculture in Life Support Systems
Nicolas Drougard, Thibault Gateau
firstname.lastname@isae.fr
Department of Aerospace vehicles design and control (DCAS)
Key words: Bioregenerative Life Support Systems (BLSS), Precision Agriculture, ALICE
project, Planning, Sequential Decision Making under Uncertainty, Reinforcement Learning (RL),
Simulator, Robotics, Data Collection, Planning Domain Description Language (PDDL), Relational Dynamic Influence Diagram Language (RDDL), Gymnasium, Python, Git.
1
Context
Now that humanity is able to go to other planets, the next challenge is to be able to settle
there permanently. Since supplying the future settlers from Earth would be far too expensive, if not impossible, one of the major problems in achieving space exploration is the
lack of resources and favorable local conditions. Moreover, according to the latest work of
the Intergovernmental Panel on Climate Change (IPCC), sustaining life on our own planet
also seems to be a challenge. Contexts that support life will be more difficult to access on
Earth, mainly because of climate change itself, and maybe in the shorter term as a consequence of the policies to fight it, and the depletion of resources. Finally, on Earth or in
space, it is necessary to find solutions to ensure the sustainability of life despite the lack
of resources and unsuitable environments.
The research in bioregenerative life support systems (BLSS) [16, 1] aims at allowing a long
term settlement of the human being in such environments. This is why ESA is interested
in the implementation of BLSS: “For more than 30 years, the European Space Agency (i.e. ESA)
is active in the field of regenerative life support system. MELiSSA (Micro-Ecological Life Support
System Alternative) is the European project of circular life support system. It was established
to gain knowledge on regenerative system, aiming to the highest degree of autonomy and consequently to produce food, water and oxygen from mission wastes.”1 . Within the framework of
1
www.melissafoundation.org
1
the MELiSSA project, design studies for greenhouses [13, 2, 17] and prototypes have been
developed (e.g. for cultivation of tuberous plants [8] in the ESA Project “Precursor of Food
Production Unit”), and many research works have been carried out (e.g. on hydroponic
systems [6] or potatoes in controlled environments [7]).
The SpaceShip FR project of the CNES (Centre National d’Etudes Spatiales), that started
in 2019 in Toulouse FRANCE, plans to build a lunar or martian type base in order to
demonstrate the solutions implemented in various fields, such as health, energy, robotics,
digital, plant cultivation and recycling. Regarding the issue of nutrition, the SpaceShip
FR project considers hydroponic greenhouse, as well as circular aquaponic system whose
fish can be used to diversify the astronauts’ daily nutrition and create fertilizers from their
waste.
Many experiments have already been carried out in space, one of the most recent being
the production of chili peppers in the ISS by NASA three years ago, which of course
also conducts a lot of research on systems growing plants for food production [4]. It was
also the first fruit in space and the longest experimentation with plants. For his part,
the ESA astronaut Thomas Pesquet has been named Food and Agriculture Organization
(FAO) Goodwill Ambassador in April 2021, on the occasion of the International Human
Space Flight Day. Finally, a french start-up has succeeded in growing lettuce in lunar
regolith. Although this experiment was not in space, it represents a big step towards
space agriculture.
2
ALICE Project
As explained and highlighted in [9, 14, 10], space crop production needs for advanced automation, robotics, modeling, and machine learning. With the support of the Innovspace
(Fablab of ISAE-SUPAERO), the ALICE (AI for Life In spaCE) project also contributes to
the research domain of Precision Agriculture in Life Suport Systems (LSS) with the general goal of using Artificial Intelligence (AI) for maximizing production and minimizing
ressource consumption. In this context, the ALICE project focuses on 4 main interrelated
issues, namely the optimization of:
1. the design of plant cultivation systems using Multi-Disciplinary Analysis and Optimization (MDAO) frameworks taking into account various models and criteria,
2. the plant and environment state estimation using Machine Learning and Computer
Vision algorithms to derive informed decisions from data,
3. the sequential decisions using Planning and (Offline) Reinforcement Learning algorithms to compute efficient and economical autonomous cultivation strategies,
4. the data transmission between systems using low power, wide area communication
protocol (LoRa).
The related studies paves the way towards autonomous plant growing systems capable
of analyzing and even reacting to the growing process in order to quickly obtain healthy
plants while optimally use space and resources (nutrient, water, energy, etc.). Twenty
students are taking part or have taken part in this project since 2019.
The ALICE project also sets up test beds to collect plant cultivation system data useful
for the 4 main issues, and to ensure that the tools developed work in practice. One of
them is a robotic hydroponic system (see Figure 1) developped and improved by previous
2
Figure 1: Robotic hydroponic system.
Figure 2: Farmbot Genesis XL robotic system.
3
students. This robot controls the intensity and the frequency of the light, as suggested by
[12, 11], the flow of the water and the nutrients level, measures the temperature, the pH
of the water, and with its robotic arm fixed on a rail, it is able to take pictures of each plant
and to move them.
Another test bed of the ALICE project is the Farmbot Genesis XL2 (see Figure 2), an
open source Cartesian coordinate robot farming machine that has been set up next to the
ISAE restaurant. This robot can plant seeds, water plants, remove weeds, take pictures,
and measure the soil moisture. The hydroponic system as well as the Farmbot may be
used for this research project addressing the third main issue, namely the optimization of the
sequential decisions using Planning and (Offline) Reinforcement Learning algorithms to compute
efficient and economical autonomous cultivation strategies.
3
Research Project objectives
The objective of the presented research project is to use Planning [3] and Reinforcement
Learning [15] to optimize the execution of plant cultivation systems.
• The first step in any research project is to begin a literature review of the subject. For
this project, it will concern plant cultivation systems, precision agriculture, planning
and reinforcement learning.
• In parallel, it will be necessary to learn how to use planning algorithms (e.g. MetricFF3 or Prost4 ) with associated languages (e.g. PDDL or RDDL), and Reinforcement
Learning algorithms trained on simulators (e.g. Gymnasium5 ) or on a large dataset
[5]. A test bed of the ALICE project (the robotic hydroponic system or the Farmbot)
will be also studied and tested, listing accessible data and possible actions of the
system on the environment, to ensure accurate modelling. The measurement tools
needed to estimate the criteria used for optimization may be added on the system.
• Once the state of the art has been sufficiently analyzed, a first planning or reinforcement learning task will be proposed, minimizing ressource consumption and maximizing production. This task will be solved and evaluated according to the chosen
criteria.
• In order to confront these research contributions with reality, the computed optimal
cultivation strategy will be executed on a test bed of the ALICE project (the robotic
hydroponic system or the Farmbot), and evaluated according to actual execution.
will then be
• The research project continues by multiplying these planning/RL tasks, improving
the associated cultivation stategies, and evaluating them in practice.
Please do not hesitate to contact us if you would like to find out more about this research project.
2
https://farm.bot
https://fai.cs.uni-saarland.de/hoffmann/metric-ff.html
4
https://github.com/prost-planner/prost
5
https://gymnasium.farama.org/index.html
3
4
References
[1] Matthew Bamsey, Daniel Schubert, Paul Zabel, Lucie Poulet, and Conrad Zeidler.
Bioregenerative life support system research as part of the dlr eden initiative. 40th
COSPAR Scientific Assembly, 40:F4–2, 2014.
[2] Katarina Eriksson, Ondrej Doule, and Lucie Poulet. Architectural concepts for a lunar
greenhouse within the melissa framework. 2014.
[3] Hector Geffner and Blai Bonet. A concise introduction to models and methods for automated planning. Springer Nature, 2022.
[4] CM Johnson, HO Boles, LE Spencer, L Poulet, M Romeyn, JM Bunchek, R Fritsche,
GD Massa, A O’Rourke, and RM Wheeler. Supplemental food production with
plants: A review of nasa research. Bioregenerative Life-Support Systems for Crewed Missions to the Moon and Mars, 2022.
[5] Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint
arXiv:2005.01643, 2020.
[6] Mariantonella Palermo, Roberta Paradiso, Stefania De Pascale, and Vincenzo
Fogliano. Hydroponic cultivation improves the nutritional quality of soybean and
its products. Journal of agricultural and food chemistry, 60(1):250–255, 2012.
[7] Roberta Paradiso, Carmen Arena, Youssef Rouphael, Luigi d’Aquino, Konstantinos
Makris, Paola Vitaglione, and Stefania De Pascale. Growth, photosynthetic activity
and tuber quality of two potato cultivars in controlled environment as affected by
light source. Plant Biosystems-An International Journal Dealing with all Aspects of Plant
Biology, 153(5):725–735, 2019.
[8] Roberta Paradiso, Antonio Ceriello, Antonio Pannico, Salvatore Sorrentino, Mario
Palladino, Maria Giordano, Raimondo Fortezza, and Stefania De Pascale. Design of a
module for cultivation of tuberous plants in microgravity: The esa project “precursor
of food production unit”(pfpu). Frontiers in Plant Science, 11:417, 2020.
[9] Lucie Poulet and Ondrej Doule. Greenhouse automation, illumination and expansion
study for mars desert research station. 2014.
[10] Lucie Poulet, Kenneth Engeling, Tyler Hatch, Sarah Stahl-Rommel, Yo-Ann Velez Justiniano, Sarah Castro-Wallace, Jess Bunchek, Oscar Monje, Mary Hummerick,
Christina LM Khodadad, et al. Large-scale crop production for the moon and mars:
current gaps and future perspectives. Frontiers in Astronomy and Space Sciences, page
226, 2022.
[11] Lucie Poulet, GD Massa, RC Morrow, CM Bourget, RM Wheeler, and CA Mitchell.
Significant reduction in energy for plant-growth lighting in space using targeted led
lighting and spectral manipulation. Life Sciences in Space Research, 2:43–53, 2014.
[12] Lucie Poulet, GD Massa, R Wheeler, T Gill, R Morrow, C Steele, T Swarmer, K Binsted, and J Hunter. Demonstration test of electrical lighting systems for plant growth
in hi-seas analog mars habitat. 2014.
5
[13] Lucie Poulet, Daniel Schubert, Conrad Zeidler, Paul Zabel, Volker Maiwald, Etienne
David, and Christel Paillé. Greenhouse modules and regenerative life-support systems for space. In AIAA SPACE 2013 Conference and Exposition, page 5398, 2013.
[14] Lucie Poulet, Conrad Zeidler, Jess Bunchek, Paul Zabel, Vincent Vrakking, Daniel
Schubert, Gioia Massa, and Raymond Wheeler. Crew time in a space greenhouse
using data from analog missions and veggie. Life Sciences in Space Research, 31:101–
112, 2021.
[15] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT
press, 2018.
[16] Cyprien Verseux, Jean-Pierre Paul de Vera, Natalie Leys, and Lucie Poulet. Bioregenerative life-support systems for crewed missions to the moon and mars. 2022.
[17] Conrad Zeidler, Vincent Vrakking, Matthew Bamsey, Lucie Poulet, Paul Zabel, Daniel
Schubert, Christel Paille, Erik Mazzoleni, and Nico Domurath. Greenhouse module
for space system: a lunar greenhouse design. Open Agriculture, 2(1):116–132, 2017.
6