ESCUELA SUPERIOR POLITÉCNICA DEL LITORAL
FACULTAD DE INGENIERÍA MECÁNICA Y CIENCIAS DE LA PRODUCCIÓN
CONTROL POR APRENDIZAJE REFORZADO
Student: Isabel Luna Ortiz
1. MDP: Chess Player Making Decisions
Description:
This MDP models the decision-making process of a chess player during a match. The agent’s
goal is to maximize the probability of winning by evaluating the current board position and the
game context at each turn. Decisions include making legal moves and are influenced by the
board state, remaining time, and material balance.
States:




Current board configuration (piece arrangement).
Current turn of the player or opponent.
The remaining time on the player’s clock.
Material balance, number of player’s pieces minus opponent’s pieces.
Actions:



Make any legal move.
Offer a draw.
Resign.
Rewards:





+100 for winning the game.
0 for a draw.
-100 for losing.
+10 for capturing a valuable piece, like rook or queen.
+5 for placing the opponent’s king in check.
2. MDP: Surveillance Drone in a Rural Area
Description:
This MDP models the behavior of an autonomous surveillance drone operating in a rural area
divided into a grid. The drone’s goal is to maximize area safety by efficiently patrolling,
detecting suspicious activity, and ensuring it returns to its base before the battery runs out. It
must adapt to changing environmental conditions such as weather, and make intelligent
decisions regarding movement and altitude, evaluating both risk and reward.
1
States:




Location in the aerial grid, coordinates x, y.
Battery level. It can be high, medium, low.
Current weather conditions. It can be clear, rainy, windy.
Presence of suspicious activity in the current cell.
Actions:




Move to an adjacent cell (North, South, East, West).
Descend for a closer inspection.
Ascend for general aerial surveillance.
Return to base for recharge and safe mission completion.
Rewards:




+50 for detecting and reporting suspicious activity.
+5 for patrolling a safe area.
-10 for inspecting during adverse weather.
-100 if the drone runs out of battery away from the base.
3. MDP: Smart Irrigation System for Crops
Description:
This MDP simulates the behavior of an automated agricultural irrigation system. The agent
must decide when and how much to irrigate, based on current soil moisture, weather forecasts,
and the crop’s growth stage. The goal is to maximize crop health while minimizing unnecessary
water usage.
States:



Current soil moisture level (low, medium, high).
Weather forecast (rain, no rain).
Crop growth stage (sowing, growing, flowering, harvesting).
Actions:




Do not irrigate.
Irrigate lightly.
Irrigate normally.
Irrigate heavily.
Rewards:




+20 for maintaining ideal moisture during a critical stage.
-15 for under-irrigating during growth or flowering stages.
-10 for over-irrigating or irrigating before rain.
-5 for maintaining poor moisture levels during non-critical stages.
2