Within the dynamic world of synthetic intelligence and tremendous development of Generative AI, builders are always searching for revolutionary methods to extract significant perception from textual content. This weblog submit walks you thru an thrilling venture that harnesses the ability of Google’s Gemini AI to create an clever English Educator Utility that analyzes textual content paperwork and offers tough phrases, medium phrases, their synonyms, antonyms, use-cases and likewise offers the essential questions with a solution from the textual content. I imagine schooling is the sector that advantages most from the developments of Generative AIs or LLMs and it’s GREAT!
Studying Targets
- Integrating Google Gemini AI fashions into Python-based APIs.
- Perceive how one can combine and make the most of the English Educator App API to boost language studying functions with real-time knowledge and interactive options.
- Learn to leverage the English Educator App API to construct custom-made instructional instruments, enhancing consumer engagement and optimizing language instruction.
- Implementation of clever textual content evaluation utilizing superior AI prompting.
- Managing advanced AI interplay error-free with error-handling strategies.
This text was printed as part of the Information Science Blogathon.
What are APIs?
API (Utility Programming Interfaces) function a digital bridge between completely different software program functions. They’re outlined as a set of protocols and guidelines that allow seamless communication, permitting builders to entry particular functionalities with out diving into advanced underlying implementation.
What’s REST API?
REST (Representational State Switch) is an architectural type for designing networked functions. It makes use of normal HTTP strategies to carry out operations on sources.
Essential REST strategies are:
- GET: Retrieve knowledge from a server.
- POST: Create new sources.
- PUT: Replace current sources fully.
- PATCH: Partially replace current sources.
- DELETE: Take away sources.
Key traits embody:
- Stateless communication
- Uniform interface
- Shopper-Serve structure
- Cacheable sources
- Layered system design
REST APIs use URLs to determine sources and usually return knowledge in JSON codecs. They supply a standardized, scalable strategy for various functions to speak over the web, making them basic in fashionable internet and cellular growth.
Pydantic and FastAPI: A Good Pair
Pydantic revolutionizes knowledge validation in Python by permitting builders to create sturdy knowledge fashions with kind hints and validation guidelines. It ensures knowledge integrity and offers crystal-clear interface definitions, catching potential errors earlier than they propagate by means of the system.
FastAPI enhances Pydantic fantastically, providing a contemporary, high-performance asynchronous internet framework for constructing APIs.
Its key benefit of FastAPI:
- Computerized interactive API documentation
- Excessive-speed efficiency
- Constructed-in assist for Asynchronous Server Gateway Interface
- Intuitive knowledge validation
- Clear and easy syntax
A Temporary on Google Gemini
Google Gemini represents a breakthrough in multimodal AI fashions, able to processing advanced data throughout textual content, code, audio, and picture. For this venture, I leverage the ‘gemini-1.5-flash’ mannequin, which offers:
- Fast and clever textual content processing utilizing prompts.
- Superior pure language understanding.
- Versatile system directions for custom-made outputs utilizing prompts.
- Potential to generate a nuanced, context-aware response.
Mission Setup and Surroundings Configuration
Organising the event surroundings is essential for a easy implementation. We use Conda to create an remoted, reproducible surroundings
# Create a brand new conda surroundings
conda create -n educator-api-env python=3.11
# Activate the surroundings
conda activate educator-api-env
# Set up required packages
pip set up "fastapi[standard]" google-generativeai python-dotenvMission Architectural Elements
Our API is structured into three major parts:
- fashions.py : Outline knowledge buildings and validation
- companies.py : Implements AI-powered textual content extractor companies
- principal.py : Create API endpoints and handles request routing
Constructing the API: Code Implementation
Getting Google Gemini API Key and Safety setup for the venture.
Create a .env file within the venture root, Seize your Gemini API Key from right here, and put your key within the .env file
GOOGLE_API_KEY="ABCDEFGH-67xGHtsf"This file might be securely accessed by the service module utilizing os.getenv(“
Pydantic Fashions: Guaranteeing Information Integrity
We outline structured fashions that assure knowledge consistency for the Gemini response. We are going to implement two knowledge fashions for every knowledge extraction service from the textual content.
Vocabulary knowledge extraction mannequin:
- WordDetails: It can construction and validate the extracted phrase from the AI
from pydantic import BaseModel, Subject
from typing import Listing, Optionally available
class WordDetails(BaseModel):
phrase: str = Subject(..., description="Extracted vocabulary phrase")
synonyms: Listing[str] = Subject(
default_factory=listing, description="Synonyms of the phrase"
)
antonyms: Listing[str] = Subject(
default_factory=listing, description="Antonyms of the phrase"
)
usecase: Optionally available[str] = Subject(None, description="Use case of the phrase")
instance: Optionally available[str] = Subject(None, description="Instance sentence")- VocabularyResponse: It can construction and validate the extracted phrases into two classes very tough phrases and medium tough phrases.
class VocabularyResponse(BaseModel):
difficult_words: Listing[WordDetails] = Subject(
..., description="Listing of inauspicious vocabulary phrases"
)
medium_words: Listing[WordDetails] = Subject(
..., description="Listing of medium vocabulary phrases"
)Query and Reply extraction mannequin
- QuestionAnswerModel: It can construction and validate the extracted questions and solutions.
class QuestionAnswerModel(BaseModel):
query: str = Subject(..., description="Query")
reply: str = Subject(..., description="Reply")- QuestionAnswerResponse: It can construction and validate the extracted responses from the AI.
class QuestionAnswerResponse(BaseModel):
questions_and_answers: Listing[QuestionAnswerModel] = Subject(
..., description="Listing of questions and solutions"
)These fashions present computerized validation, kind checking, and clear interface definitions, stopping potential runtime errors.
Service Module: Clever Textual content Processing
The service module has two companies:
This service GeminiVocabularyService services:
- Makes use of Gemini to determine difficult phrases.
- Generates complete phrase insights.
- Implement sturdy JSON parsing.
- Manages potential error situations.
First, we’ve to import all the mandatory libraries and arrange the logging and surroundings variables.
import os
import json
import logging
from fastapi import HTTPException
import google.generativeai as genai
from dotenv import load_dotenv
# Configure logging
logging.basicConfig(degree=logging.INFO)
logger = logging.getLogger(__name__)
# Load surroundings variables
load_dotenv()This GeminiVocabularyService class has three methodology.
The __init__ Technique has essential Gemini configuration, Google API Key, Setting generative mannequin, and immediate for the vocabulary extraction.
Immediate:
"""You're an knowledgeable vocabulary extractor.
For the given textual content:
1. Determine 3-5 difficult vocabulary phrases
2. Present the next for EACH phrase in a STRICT JSON format:
- phrase: The precise phrase
- synonyms: Listing of 2-3 synonyms
- antonyms: Listing of 2-3 antonyms
- usecase: A quick rationalization of the phrase's utilization
- instance: An instance sentence utilizing the phrase
IMPORTANT: Return ONLY a legitimate JSON that matches this construction:
{
"difficult_words": [
{
"word": "string",
"synonyms": ["string1", "string2"],
"antonyms": ["string1", "string2"],
"usecase": "string",
"instance": "string"
}
],
"medium_words": [
{
"word": "string",
"synonyms": ["string1", "string2"],
"antonyms": ["string1", "string2"],
"usecase": "string",
"instance": "string"
}
],
}
""" Code Implementation
class GeminiVocabularyService:
def __init__(self):
_google_api_key = os.getenv("GOOGLE_API_KEY")
# Retrieve API Key
self.api_key = _google_api_key
if not self.api_key:
elevate ValueError(
"Google API Key's lacking. Please set GOOGLE_API_KEY in .env file."
)
# Configure Gemini API
genai.configure(api_key=self.api_key)
# Technology Configuration
self.generation_config = {
"temperature": 0.7,
"top_p": 0.95,
"max_output_tokens": 8192,
}
# Create Generative Mannequin
self.vocab_model = genai.GenerativeModel(
model_name="gemini-1.5-flash",
generation_config=self.generation_config, # kind: ignore
system_instruction="""
You're an knowledgeable vocabulary extractor.
For the given textual content:
1. Determine 3-5 difficult vocabulary phrases
2. Present the next for EACH phrase in a STRICT JSON format:
- phrase: The precise phrase
- synonyms: Listing of 2-3 synonyms
- antonyms: Listing of 2-3 antonyms
- usecase: A quick rationalization of the phrase's utilization
- instance: An instance sentence utilizing the phrase
IMPORTANT: Return ONLY a legitimate JSON that matches this construction:
{
"difficult_words": [
{
"word": "string",
"synonyms": ["string1", "string2"],
"antonyms": ["string1", "string2"],
"usecase": "string",
"instance": "string"
}
],
"medium_words": [
{
"word": "string",
"synonyms": ["string1", "string2"],
"antonyms": ["string1", "string2"],
"usecase": "string",
"instance": "string"
}
],
}
""",
)The extracted_vocabulary methodology has the chat course of, and response from the Gemini by sending textual content enter utilizing the sending_message_async() operate. This methodology has one non-public utility operate _parse_response(). This non-public utility operate will validate the response from the Gemini, verify the mandatory parameters then parse the info to the extracted vocabulary operate. It can additionally log the errors reminiscent of JSONDecodeError, and ValueError for higher error administration.
Code Implementation
The extracted_vocabulary methodology:
async def extract_vocabulary(self, textual content: str) -> dict:
strive:
# Create a brand new chat session
chat_session = self.vocab_model.start_chat(historical past=[])
# Ship message and await response
response = await chat_session.send_message_async(textual content)
# Extract and clear the textual content response
response_text = response.textual content.strip()
# Try to extract JSON
return self._parse_response(response_text)
besides Exception as e:
logger.error(f"Vocabulary extraction error: {str(e)}")
logger.error(f"Full response: {response_text}")
elevate HTTPException(
status_code=500, element=f"Vocabulary extraction failed: {str(e)}"
)The _parsed_response methodology:
def _parse_response(self, response_text: str) -> dict:
# Take away markdown code blocks if current
response_text = response_text.exchange("```json", "").exchange("```", "").strip()
strive:
# Try to parse JSON
parsed_data = json.hundreds(response_text)
# Validate the construction
if (
not isinstance(parsed_data, dict)
or "difficult_words" not in parsed_data
):
elevate ValueError("Invalid JSON construction")
return parsed_data
besides json.JSONDecodeError as json_err:
logger.error(f"JSON Decode Error: {json_err}")
logger.error(f"Problematic response: {response_text}")
elevate HTTPException(
status_code=400, element="Invalid JSON response from Gemini"
)
besides ValueError as val_err:
logger.error(f"Validation Error: {val_err}")
elevate HTTPException(
status_code=400, element="Invalid vocabulary extraction response"
)The entire CODE of the GeminiVocabularyService module.
class GeminiVocabularyService:
def __init__(self):
_google_api_key = os.getenv("GOOGLE_API_KEY")
# Retrieve API Key
self.api_key = _google_api_key
if not self.api_key:
elevate ValueError(
"Google API Key's lacking. Please set GOOGLE_API_KEY in .env file."
)
# Configure Gemini API
genai.configure(api_key=self.api_key)
# Technology Configuration
self.generation_config = {
"temperature": 0.7,
"top_p": 0.95,
"max_output_tokens": 8192,
}
# Create Generative Mannequin
self.vocab_model = genai.GenerativeModel(
model_name="gemini-1.5-flash",
generation_config=self.generation_config, # kind: ignore
system_instruction="""
You're an knowledgeable vocabulary extractor.
For the given textual content:
1. Determine 3-5 difficult vocabulary phrases
2. Present the next for EACH phrase in a STRICT JSON format:
- phrase: The precise phrase
- synonyms: Listing of 2-3 synonyms
- antonyms: Listing of 2-3 antonyms
- usecase: A quick rationalization of the phrase's utilization
- instance: An instance sentence utilizing the phrase
IMPORTANT: Return ONLY a legitimate JSON that matches this construction:
{
"difficult_words": [
{
"word": "string",
"synonyms": ["string1", "string2"],
"antonyms": ["string1", "string2"],
"usecase": "string",
"instance": "string"
}
],
"medium_words": [
{
"word": "string",
"synonyms": ["string1", "string2"],
"antonyms": ["string1", "string2"],
"usecase": "string",
"instance": "string"
}
],
}
""",
)
async def extract_vocabulary(self, textual content: str) -> dict:
strive:
# Create a brand new chat session
chat_session = self.vocab_model.start_chat(historical past=[])
# Ship message and await response
response = await chat_session.send_message_async(textual content)
# Extract and clear the textual content response
response_text = response.textual content.strip()
# Try to extract JSON
return self._parse_response(response_text)
besides Exception as e:
logger.error(f"Vocabulary extraction error: {str(e)}")
logger.error(f"Full response: {response_text}")
elevate HTTPException(
status_code=500, element=f"Vocabulary extraction failed: {str(e)}"
)
def _parse_response(self, response_text: str) -> dict:
# Take away markdown code blocks if current
response_text = response_text.exchange("```json", "").exchange("```", "").strip()
strive:
# Try to parse JSON
parsed_data = json.hundreds(response_text)
# Validate the construction
if (
not isinstance(parsed_data, dict)
or "difficult_words" not in parsed_data
):
elevate ValueError("Invalid JSON construction")
return parsed_data
besides json.JSONDecodeError as json_err:
logger.error(f"JSON Decode Error: {json_err}")
logger.error(f"Problematic response: {response_text}")
elevate HTTPException(
status_code=400, element="Invalid JSON response from Gemini"
)
besides ValueError as val_err:
logger.error(f"Validation Error: {val_err}")
elevate HTTPException(
status_code=400, element="Invalid vocabulary extraction response"
)
Query-Reply Technology Service
This Query Reply Service services:
- Creates contextually wealthy comprehension questions.
- Generates exact, informative solutions.
- Handles advanced textual content evaluation requirement.
- JSON and Worth error dealing with.
This QuestionAnswerService has three strategies:
__init__ methodology
The __init__ methodology is generally the identical because the Vocabulary service class aside from the immediate.
Immediate:
"""
You're an knowledgeable at creating complete comprehension questions and solutions.
For the given textual content:
1. Generate 8-10 numerous questions overlaying:
- Vocabulary that means
- Literary units
- Grammatical evaluation
- Thematic insights
- Contextual understanding
IMPORTANT: Return ONLY a legitimate JSON on this EXACT format:
{
"questions_and_answers": [
{
"question": "string",
"answer": "string"
}
]
}
Tips:
- Questions ought to be clear and particular
- Solutions ought to be concise and correct
- Cowl completely different ranges of comprehension
- Keep away from sure/no questions
""" Code Implementation:
The __init__ methodology of QuestionAnswerService class
def __init__(self):
_google_api_key = os.getenv("GOOGLE_API_KEY")
# Retrieve API Key
self.api_key = _google_api_key
if not self.api_key:
elevate ValueError(
"Google API Key's lacking. Please set GOOGLE_API_KEY in .env file."
)
# Configure Gemini API
genai.configure(api_key=self.api_key)
# Technology Configuration
self.generation_config = {
"temperature": 0.7,
"top_p": 0.95,
"max_output_tokens": 8192,
}
self.qa_model = genai.GenerativeModel(
model_name="gemini-1.5-flash",
generation_config=self.generation_config, # kind: ignore
system_instruction="""
You're an knowledgeable at creating complete comprehension questions and solutions.
For the given textual content:
1. Generate 8-10 numerous questions overlaying:
- Vocabulary that means
- Literary units
- Grammatical evaluation
- Thematic insights
- Contextual understanding
IMPORTANT: Return ONLY a legitimate JSON on this EXACT format:
{
"questions_and_answers": [
{
"question": "string",
"answer": "string"
}
]
}
Tips:
- Questions ought to be clear and particular
- Solutions ought to be concise and correct
- Cowl completely different ranges of comprehension
- Keep away from sure/no questions
""",
)The Query and Reply Extraction
The extract_questions_and_answers methodology has a chat session with Gemini, a full immediate for higher extraction of questions and solutions from the enter textual content, an asynchronous message despatched to the Gemini API utilizing send_message_async(full_prompt), after which stripping response knowledge for clear knowledge. This methodology additionally has a personal utility operate similar to the earlier one.
Code Implementation:
extract_questions_and_answers
async def extract_questions_and_answers(self, textual content: str) -> dict:
"""
Extracts questions and solutions from the given textual content utilizing the supplied mannequin.
"""
strive:
# Create a brand new chat session
chat_session = self.qa_model.start_chat(historical past=[])
full_prompt = f"""
Analyze the next textual content and generate complete comprehension questions and solutions:
{textual content}
Make sure the questions and solutions present deep insights into the textual content's that means, type, and context.
"""
# Ship message and await response
response = await chat_session.send_message_async(full_prompt)
# Extract and clear the textual content response
response_text = response.textual content.strip()
# Try to parse and validate the response
return self._parse_response(response_text)
besides Exception as e:
logger.error(f"Query and reply extraction error: {str(e)}")
logger.error(f"Full response: {response_text}")
elevate HTTPException(
status_code=500, element=f"Query-answer extraction failed: {str(e)}"
)_parse_response
def _parse_response(self, response_text: str) -> dict:
"""
Parses and validates the JSON response from the mannequin.
"""
# Take away markdown code blocks if current
response_text = response_text.exchange("```json", "").exchange("```", "").strip()
strive:
# Try to parse JSON
parsed_data = json.hundreds(response_text)
# Validate the construction
if (
not isinstance(parsed_data, dict)
or "questions_and_answers" not in parsed_data
):
elevate ValueError("Response have to be an inventory of questions and solutions.")
return parsed_data
besides json.JSONDecodeError as json_err:
logger.error(f"JSON Decode Error: {json_err}")
logger.error(f"Problematic response: {response_text}")
elevate HTTPException(
status_code=400, element="Invalid JSON response from the mannequin"
)
besides ValueError as val_err:
logger.error(f"Validation Error: {val_err}")
elevate HTTPException(
status_code=400, element="Invalid question-answer extraction response"
)API Endpoints: Connecting Customers to AI
The primary file defines two major POST endpoint:
It’s a submit methodology that may primarily devour enter knowledge from the shoppers and ship it to the AI APIs by means of the vocabulary Extraction Service. It can additionally verify the enter textual content for minimal phrase necessities and in spite of everything, it is going to validate the response knowledge utilizing the Pydantic mannequin for consistency and retailer it within the storage.
@app.submit("/extract-vocabulary/", response_model=VocabularyResponse)
async def extract_vocabulary(textual content: str):
# Validate enter
if not textual content or len(textual content.strip()) This submit methodology, could have largely the identical because the earlier POST methodology besides it is going to use Query Reply Extraction Service.
@app.submit("/extract-question-answer/", response_model=QuestionAnswerResponse)
async def extract_question_answer(textual content: str):
# Validate enter
if not textual content or len(textual content.strip()) There are two major GET methodology:
First, the get-vocabulary methodology will verify the hash key with the shoppers’ textual content knowledge, if the textual content knowledge is current within the storage the vocabulary might be offered as JSON knowledge. This methodology is used to indicate the info on the CLIENT SIDE UI on the internet web page.
@app.get("/get-vocabulary/", response_model=Optionally available[VocabularyResponse])
async def get_vocabulary(textual content: str):
"""
Retrieve the vocabulary response for a beforehand processed textual content.
"""
key = hash(textual content)
if key in vocabulary_storage:
return vocabulary_storage[key]
else:
elevate HTTPException(
status_code=404, element="Vocabulary consequence not discovered for the supplied textual content"
)Second, the get-question-answer methodology may even verify the hash key with the shoppers’ textual content knowledge similar to the earlier methodology, and can produce the JSON response saved within the storage to the CLIENT SIDE UI.
@app.get("/get-question-answer/", response_model=Optionally available[QuestionAnswerResponse])
async def get_question_answer(textual content: str):
"""
Retrieve the question-answer response for a beforehand processed textual content.
"""
key = hash(textual content)
if key in qa_storage:
return qa_storage[key]
else:
elevate HTTPException(
status_code=404,
element="Query-answer consequence not discovered for the supplied textual content",
)Key Implementation Function
To run the applying, we’ve to import the libraries and instantiate a FastAPI service.
Import Libraries
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from typing import Optionally available
from .fashions import VocabularyResponse, QuestionAnswerResponse
from .companies import GeminiVocabularyService, QuestionAnswerServiceInstantiate FastAPI Utility
# FastAPI Utility
app = FastAPI(title="English Educator API")Cross-origin Useful resource Sharing (CORS) Help
Cross-origin useful resource sharing (CORS) is an HTTP-header-based mechanism that permits a server to point any origins reminiscent of area, scheme, or port aside from its personal from which a browser ought to allow loading sources. For safety causes, the browser restricts CORS HTTP requests initiated from scripts.
# FastAPI Utility
app = FastAPI(title="English Educator API")
# Add CORS middleware
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)In-memory validation mechanism: Easy Key Phrase Storage
We use easy key-value-based storage for the venture however you should utilize MongoDB.
# Easy key phrase storage
vocabulary_storage = {}
qa_storage = {}Enter Validation mechanisms and Complete error dealing with.
Now’s the time to run the applying.
To run the applying in growth mode, we’ve to make use of FasyAPI CLI which can put in with the FastAPI.
Sort the code to your terminal within the software root.
$ fastapi dev principal.pyOutput:
Then should you CTRL + Proper Click on on the hyperlink http://127.0.0.1:8000 you’re going to get a welcome display screen on the internet browser.
To go to the docs web page of FastAPI simply click on on the following URL or kind http://127.0.0.1:8000/docs in your browser, and you will notice all of the HTTP strategies on the web page for testing.
Now to check the API, Click on on any of the POST strategies and TRY IT OUT, put any textual content you wish to within the enter subject, and execute. You’ll get the response in response to the companies reminiscent of vocabulary, and query reply.
Execute:
Response:
Execute:
Response:
Testing Get Strategies
Get vocabulary from the storage.
Execute:
Put the identical textual content you placed on the POST methodology on the enter subject.
Response:
You’ll get the beneath output from the storage.
and likewise for question-and-answer
Execute:
Response:
That might be absolutely working internet server API for English educators utilizing Google Gemini AI.
Additional Growth Alternative
The present implementation opens doorways to thrilling future enhancements:
- Discover persistent storage options to retain knowledge successfully throughout classes.
- Combine sturdy authentication mechanisms for enhanced safety.
- Advance textual content evaluation capabilities with extra refined options.
- Design and construct an intuitive front-end interface for higher consumer interplay.
- Implement environment friendly fee limiting and caching methods to optimize efficiency.
Sensible Issues and Limitations
Whereas our API demonstrates highly effective capabilities, you must take into account:
- Contemplate API utilization prices and fee limits when planning utilization to keep away from surprising prices and guarantee scalability.
- Be aware of processing time for advanced texts, as longer or intricate inputs might end in slower response occasions.
- Put together for steady mannequin updates from Google, which can impression the API’s efficiency or capabilities over time.
- Perceive that AI-generated responses can fluctuate, so it’s essential to account for potential inconsistencies in output high quality.
Conclusion
We have now created a versatile, clever API that transforms textual content evaluation by means of the synergy of Google Gemini, FastAPI, and Pydantic. This answer demonstrates how fashionable AI applied sciences may be leveraged to extract deep, significant insights from textual knowledge.
You may get all of the code of the venture within the CODE REPO.
Key Takeaways
- AI-powered APIs can present clever, context-aware textual content evaluation.
- FastAPI simplifies advanced API growth with computerized documentation.
- The English Educator App API empowers builders to create interactive and personalised language studying experiences.
- Integrating the English Educator App API can streamline content material supply, enhancing each instructional outcomes and consumer engagement.
Incessantly Requested Query
A. The present model makes use of environment-based API key administration and contains basic enter validation. For manufacturing, extra safety layers are advisable.
A. All the time assessment Google Gemini’s present phrases of service and licensing for business implementations.
A. Efficiency will depend on Gemini API response occasions, enter complexity, and your particular processing necessities.
A. The English Educator App API offers instruments for educators to create personalised language studying experiences, providing options like vocabulary extraction, pronunciation suggestions, and superior textual content evaluation.
The media proven on this article is just not owned by Analytics Vidhya and is used on the Creator’s discretion.
A self-taught, project-driven learner, like to work on advanced tasks on deep studying, Pc imaginative and prescient, and NLP. I at all times attempt to get a deep understanding of the subject which can be in any subject reminiscent of Deep studying, Machine studying, or Physics. Like to create content material on my studying. Attempt to share my understanding with the worlds.
