KCAU SCHOOL OF TECHNOLOGY
BSD 3205 EMBEDDED SYSTEMS
DR. LAWRENCE NDERU
AMBIENT ENVIRONMENT MANIPULATOR
WITH ARDUINO
Introduction
The system revolves around the fact that we want some change in the environment we are in
according to what we have going on in our personal computers. Personal computers are considered
as we mainly live in our screens. This system uses arduino to enable those changes depending on
the open window on the computer.
We will use a python script that will start running on pc wake, to detect the currently open window
on the pc. Depending on the window, the script will produce an output that will be transferred to
the arduino board as input using serial communication. The arduino board will be programmed to
do specific actions based on the output of the script which is input to the board.
This can be used to automate multiple actions but for this specific project we will be automating
the task of changing the room lighting and tourning on speaker on window open. The board will
lower the room lighting in the event that a video player application such as vlc or netflix is open
and increase when a study application such as adobe pdf viewer or text editor is open. Bluetooth
speaker will be automatically turned on when a music or video player is opened. The specific tasks
can also be done via a voice command where the arduino will act as if a video player is open if the
person says the word ‘movie’ in their command, ‘music’ for audio and ‘study’ for text/document
editors. The speech is activated by an ‘activate’ command and a ‘sleep’ command can be used to
shut everything down. After the activate command then commands can be taken to do what the
user requires.
SYSTEM FEATURES
components Quantity
Arduino uno
Laptop
speaker
Led
Jumper cables
Switch
Resistors
Breadboard
1
1
1
3
Price
2
5
1
software:





os(windows 10/11 or debian/ubuntu based linux)
arduino ide
vs code
video/music player(vlc & spotify)
text editor/ writer app(libre office, ms word, adobe pdf viewer, notepad)
skills needed

Basic electronics knowledge: The project will involve wiring and connecting various
components using jumper wires and a breadboard. Basic knowledge of electronics is
necessary to ensure the components are connected correctly and safely.

Familiarity with python scripting (Franklin koome – lead developer) and c programming
language. The project will involve writing and uploading code to the Arduino board using
the Arduino IDE. After the sketch is uploaded the arduino ide’s communication to the board
is stopped and the python script starts sending data.

Understanding of media sensor technology: The project will involve using a media sensor/
action trigger . A basic understanding of how the sensor works and how it detects media
activity is necessary to program the Arduino to respond to the sensor input correctly.

Problem-solving and critical thinking:(whole group) As with any DIY project, it is likely
that unexpected issues will arise during the build process. The ability to troubleshoot and
find solutions is important to the success of the project.
Reading and understanding technical documentation: (Leon karani - documentation) The
project will involve using technical documentation to understand how to connect the

components and write the code. The ability to read and understand technical documentation
is necessary to ensure that the project is completed correctly and safely.
Existing skills
 Familiarity with python programming (Franklin koome – lead developer)
 Reading and understanding technical documentation: (Leon karani - documentation)
 Problem-solving and critical thinking:()
 Research (Anne Nzisa)
skills to acquire
 Basic electronics knowledge
 Understanding of Arduino technology
 Familiarity with Arduino programming (c programming language)
 Integration of python and arduino
languages:
Arduino’s c programming
Python scripting
Schematics:
Python code for a Debian based Linux os(debian_project.py):
#!/usr/bin/env python
import subprocess
import time
import serial
import threading
import speech_recognition as sr
from Applications import *
import pyttsx3
# Set the initial window ID and name
prev_window_id = None
prev_window_name = None
prev_command = "sleep"
command = "sleep"
# Create a lock object
lock = threading.Lock()
def voice_out(command):
# Initialize the engine
engine = pyttsx3.init()
engine.setProperty('rate', 150)
engine.say(command)
engine.runAndWait()
# Define a function to send data to the Arduino
def send_data(data):
try:
ser = serial.Serial("/dev/ttyACM0", 9600)
ser.write(data.encode()) # send the data to arduino
except FileNotFoundError:
print("No Device Found\nExiting ...")
exit()
except serial.serialutil.SerialException as e:
print(f"Error: {e}")
# Define a function to check the focused window and send data to Arduino
def process_open_apps():
global prev_window_id, prev_command, command
while True:
# Run the xdotool command to get the active window ID
result = subprocess.run(['xdotool', 'getactivewindow'],
capture_output=True, text=True)
# Extract the window ID from the command output
window_id = result.stdout.strip()
# Run the xdotool command to get the window name for the active window
result = subprocess.run(['xdotool', 'getwindowname', window_id],
capture_output=True, text=True)
# Extract the window name from the command output
window_name = result.stdout.strip()
# Check if the window focus has changed
if window_id != prev_window_id:
# loop through the various data lists
for audio_player, video_player, browser, text_editor, document_app
in zip(audio_players, video_players,
browser_apps, text_ediitors,
document_file_apps):
# audio player check
if audio_player in window_name.lower():
command = "audio"
# video player check
elif video_player in window_name.lower():
command = "video"
# browser check
elif browser in window_name.lower():
command = "browser"
# text editor/coding apps
elif text_editor in window_name.lower():
command = "code"
# Document editors/viewers check
elif document_app in window_name.lower():
command = "document"
# Browser check
elif browser in window_name.lower():
command = "browser"
if prev_command != command:
# Acquire the lock before sending data to the Arduino
if command == "audio":
voice_out("Adjusting environment for music")
elif command == "video":
voice_out("Adjusting environment for watching")
elif command == "browser":
voice_out("Adjusting environment for browsing")
elif command == "code":
voice_out("Adjusting environment for coding")
elif command == "document":
voice_out("Adjusting environment for studying")
lock.acquire()
send_data(command)
lock.release()
# Write the window open and time in a text file
with open("opened_apps.txt", 'a') as file:
file.write(f"Program: {command}\nTime: {time.ctime()}\n")
prev_command = command # update the previous data
# Update the previous window ID and name
prev_window_id = window_id
# Wait for some time before checking again
time.sleep(1)
# Define a function to process speech input and send data to Arduino
def process_speech():
global command
# initialize the recognizer
r = sr.Recognizer()
mic = sr.Microphone()
trigger_phrase = "activate"
shutdown_phrase = "sleep"
# keep the recognizer on in an indefinite loop
while True:
# open the microphone to listen for input
with mic as source:
print("Waiting for Trigger Phrase...")
r.adjust_for_ambient_noise(source) # remove noise
audio = r.listen(source, phrase_time_limit=5)
# try recognizing the input using google API
try:
print("Recognizing...")
activator = r.recognize_google(audio, language='en-in')
if trigger_phrase in activator.lower():
voice_out("Activated, waiting for command.")
with mic as source:
print("Activated\nListening...")
r.adjust_for_ambient_noise(source) # remove noise
command = r.listen(source, phrase_time_limit=5)
try:
query = r.recognize_google(command)
except sr.exceptions.UnknownValueError:
voice_out("No command taken")
except sr.exceptions.RequestError:
pass
else:
if "lights" in query.lower() and "on" in query.lower():
command = "light on"
elif "lights" in query.lower() and "off" in query.lower():
command = "light off"
elif "speaker" in query.lower() and "on" in query.lower():
command = "speaker on"
elif "speaker" in query.lower() and "off" in query.lower():
command = "speaker off"
elif "movie" in query.lower():
command = "video"
voice_out("Adjusting environment for watching")
# subprocess.Popen("vlc")
elif "music" in query.lower():
command = "audio"
voice_out("Adjusting environment for music")
# subprocess.Popen("audacious")
elif "code" in query.lower() or "coding" in query.lower():
command = "code"
voice_out("Adjusting environment for coding")
elif "study" in query.lower():
command = "document"
voice_out("Adjusting environment for studying")
elif "browse" in query.lower():
command = "browser"
voice_out("Adjusting environment for browsing")
# subprocess.Popen("mousepad")
print(command + " sent")
send_data(command)
elif shutdown_phrase in activator.lower():
voice_out("Sleeping")
send_data(shutdown_phrase) # Send a sleep command to arduino
exit()
# catch any errors from the recognizer
except (sr.exceptions.RequestError, sr.exceptions.UnknownValueError):
pass
time.sleep(1)
def main():
# Create two threads for each function
open_apps_thread = threading.Thread(target=process_open_apps)
speech_thread = threading.Thread(target=process_speech)
# Start both threads
open_apps_thread.start()
speech_thread.start()
# Wait for both threads to finish
open_apps_thread.join()
speech_thread.join()
if __name__ == '__main__':
main()
Arduino’s code to upload to board (project.h):
const int SPEAKER_BUTTON_PIN = 2;
const int LED_BUTTON_PIN = 3;
const int LED_PIN = 4;
const int SPEAKER_PIN = 5;
bool speakerOn = false;
bool ledOn = false;
String received_command;
void setup() {
pinMode(SPEAKER_PIN, OUTPUT);
pinMode(SPEAKER_BUTTON_PIN, INPUT_PULLUP);
pinMode(LED_BUTTON_PIN, INPUT_PULLUP);
pinMode(LED_PIN, OUTPUT);
Serial.begin(9600);
}
void loop() {
if (digitalRead(SPEAKER_BUTTON_PIN) == LOW) {
digitalWrite(SPEAKER_PIN, !speakerOn); // Toggle the speaker state
speakerOn = !speakerOn;
delay(500); // Debounce delay
}
// Check if the LED button is pressed
if (digitalRead(LED_BUTTON_PIN) == LOW) {
digitalWrite(LED_PIN, !ledOn); // Toggle the LED state
ledOn = !ledOn;
delay(500); // Debounce delay
}
if (Serial.available() > 0) {
received_command = Serial.readStringUntil("\n");
received_command.trim();
if (received_command == "browser") {
analogWrite(LED_PIN, 204);
ledOn = true;
delay(500);
}
else if (received_command == "light on") {
digitalWrite(LED_PIN, HIGH);
ledOn = true;
delay(500);
}
else if (received_command == "speaker on") {
digitalWrite(SPEAKER_PIN, HIGH);
speakerOn = true;
delay(1500);
}
else if (received_command == "light off") {
digitalWrite(LED_PIN, LOW);
ledOn = false;
delay(500);
}
}
}
else if (received_command == "speaker on") {
digitalWrite(SPEAKER_PIN, LOW);
speakerOn = false;
delay(1500);
}
else if (received_command == "audio") {
analogWrite(LED_PIN, 204);
ledOn = true;
delay(500);
if (!speakerOn) {
digitalWrite(SPEAKER_PIN, HIGH);
delay(3000);
speakerOn = true;
}
}
else if (received_command == "video") {
digitalWrite(LED_BUTTON_PIN, HIGH);
analogWrite(LED_PIN, 102);
ledOn = true;
if (!speakerOn) {
digitalWrite(SPEAKER_PIN, HIGH);
delay(2500);
speakerOn = true;
}
}
else if (received_command == "document" || received_command == "code") {
digitalWrite(SPEAKER_PIN, LOW);
speakerOn = false;
digitalWrite(LED_PIN, HIGH);
ledOn = true;
delay(500);
}
else if (received_command == "sleep") {
digitalWrite(SPEAKER_PIN, LOW);
speakerOn = false;
digitalWrite(LED_PIN, LOW);
ledOn = false;
delay(500);
}