Python Exercise – Module 2
Objective:
This assignment explores the topics covered in Module 2, such as natural language
structure, syntax, semantics, POS tagging, and named entity recognition. Students will apply
Python-based NLP tools to extract structured information from free-form text.
Tools:
Python (Jupyter Notebook or IDE), pandas, nltk, spacy, matplotlib, networkx
Dataset:
linguistic_annotation_dataset.csv (provided with this assignment)
Setup Instructions:
1. Install required libraries:
pip install pandas nltk spacy matplotlib networkx
2. Download language model for spacy (if using spacy):
python -m spacy download en_core_web_sm
3. Load the dataset:
df = pd.read_csv("linguistic_annotation_dataset.csv")
print(df.head())
Exercise Tasks
Part A: Corpus Exploration (10 points)
1. Load the dataset and display the first few rows.
2. Compute statistics like average sentence length and vocabulary size.
Part B: Text Preprocessing (15 points)
1. Perform sentence segmentation and word tokenization.
2. Normalize text by removing punctuation and converting to lowercase.
Part C: Part-of-Speech Tagging (25 points)
1. Apply POS tagging to each sentence.
2. Display POS-tagged sentences for the first 10 entries.
3. Visualize frequency distribution of major POS categories.
Part D: Named Entity Recognition (NER) (25 points)
1. Apply NER to detect PERSON, ORG, GPE, and DATE entities.
2. Display extracted entities and count frequency by type.
3. Present results in a tabular format.
Part E: Semantic Network Construction (Advanced – 25 points)
1. Identify co-occurring named entities within the same sentence.
2. Construct a semantic co-occurrence network.
3. Visualize it using NetworkX and Matplotlib.
Submission Instructions
- Submit a Jupyter Notebook (.ipynb) or Python script (.py) with your completed code.
- Include brief markdown or text explanations for each section.
- File name: LastName_FirstName_Module2_Python_Exercise.ipynb