Listen to this Post
Introduction:
The intersection of artificial intelligence and healthcare has long promised to make medical information more accessible, yet few projects deliver on this promise with the depth and cultural sensitivity of KhimiYaar AI. Built by Aadil Kamal Khan, this AI-assisted health information platform combines conversational AI, computer vision, and machine learning into a single unified system—featuring an AI Assistant, Lab Report Analyzer, X-ray Fracture Screening, and emergency detection with region-specific contact information. What sets KhimiYaar AI apart is its commitment to usability, transparency, and responsible AI, all delivered through a modern tech stack of React, FastAPI, and TensorFlow, with support for both English and Roman Urdu.
Learning Objectives:
- Understand the architectural components and integration patterns for building a multi-modal AI healthcare platform using React, FastAPI, and TensorFlow.
- Learn how to implement lab report analysis using NLP and computer vision techniques for extracting and interpreting medical data from PDFs and images.
- Master the deployment of CNN-based fracture detection models for X-ray image classification, including data preprocessing, augmentation, and model serving.
You Should Know:
- Architecting a Multi-Modal AI Healthcare Platform with React, FastAPI, and TensorFlow
KhimiYaar AI exemplifies a modern full-stack AI architecture where the frontend and backend operate in seamless harmony. The platform uses React for the user interface, providing a responsive and interactive experience for users uploading lab reports, chatting with the AI assistant, or submitting X-ray images for fracture screening. The backend is powered by FastAPI, a high-performance Python web framework that serves as the core processing engine for machine learning predictions. FastAPI handles prediction requests from the frontend, processes input data using trained TensorFlow models, and returns prediction results with low latency—critical for a healthcare application where response time can impact user experience.
The TensorFlow ML backend loads pre-trained models from the file system, making it an internal service that can be scaled independently. This decoupled architecture allows for separate development cycles: the frontend team can iterate on UI/UX while the ML team refines model accuracy without disrupting the entire system. For deployment, the platform can be containerized using Docker and orchestrated with Kubernetes, enabling horizontal scaling during peak usage.
Step-by-Step Guide: Setting Up the FastAPI ML Backend
1. Create a Python virtual environment
python -m venv venv
source venv/bin/activate On Windows: venv\Scripts\activate
<ol>
<li>Install dependencies
pip install fastapi uvicorn tensorflow pillow numpy python-multipart</p></li>
<li><p>Create the FastAPI application (main.py)
from fastapi import FastAPI, File, UploadFile
import tensorflow as tf
import numpy as np
from PIL import Image
import io</p></li>
</ol>
<p>app = FastAPI()
Load pre-trained model
model = tf.keras.models.load_model('models/fracture_detection.h5')
@app.post("/predict/fracture")
async def predict_fracture(file: UploadFile = File(...)):
Read and preprocess image
contents = await file.read()
image = Image.open(io.BytesIO(contents)).resize((224, 224))
image_array = np.array(image) / 255.0
image_array = np.expand_dims(image_array, axis=0)
Run prediction
prediction = model.predict(image_array)
result = "Fracture Detected" if prediction[bash][0] > 0.5 else "No Fracture"
confidence = float(prediction[bash][0])
return {"result": result, "confidence": confidence}
<ol>
<li>Run the server
uvicorn main:app --host 0.0.0.0 --port 8000 --reload
This setup creates a RESTful API endpoint that accepts X-ray image uploads, processes them through a TensorFlow CNN model, and returns a prediction with confidence score. The `–reload` flag enables hot-reloading during development, while production deployments would use Gunicorn with Uvicorn workers for better performance.
- Implementing Lab Report Analysis with NLP and Computer Vision
The Lab Report Analyzer in KhimiYaar AI addresses one of the most common barriers to healthcare access: the inability to understand complex medical reports. This feature combines optical character recognition (OCR) for extracting text from uploaded report images or PDFs, with natural language processing (NLP) for interpreting the extracted data.
The system automatically extracts key health parameters, calculates health scores, and performs risk analysis for conditions like diabetes, heart conditions, and anemia. By leveraging a Retrieval-Augmented Generation (RAG) methodology, the platform enables intelligent querying of patient lab results, allowing users to ask questions like “What does my elevated glucose level mean?” and receive contextual, easy-to-understand explanations.
Step-by-Step Guide: Building a Lab Report Analyzer
Using Tesseract OCR and Transformers for report analysis
import pytesseract
from PIL import Image
import pdf2image
from transformers import pipeline
<ol>
<li>Extract text from uploaded report
def extract_text_from_pdf(pdf_path):
images = pdf2image.convert_from_path(pdf_path)
text = ""
for image in images:
text += pytesseract.image_to_string(image)
return text</li>
</ol>
def extract_text_from_image(image_path):
image = Image.open(image_path)
return pytesseract.image_to_string(image)
<ol>
<li>Analyze extracted text with a medical NLP model
def analyze_lab_report(text):
Initialize a medical-specific NER pipeline
ner_pipeline = pipeline("ner", model="d4data/biomedical-1er-all")
entities = ner_pipeline(text)
Extract key health metrics
metrics = {}
for entity in entities:
if entity['entity'] in ['LABEL_0', 'LABEL_1']: Custom entity types
metrics[entity['word']] = entity['score']
Perform risk analysis
risk_analysis = {
"glucose": "High" if "glucose" in text and "> 140" in text else "Normal",
"cholesterol": "Elevated" if "LDL" in text and "> 130" in text else "Normal"
}
return {"metrics": metrics, "risks": risk_analysis}
For production-grade implementations, platforms like HealthPort AI use Gemini API for PDF analysis, while others leverage AWS Bedrock with LangChain for document processing. The choice depends on privacy requirements, cost considerations, and the need for offline capability.
3. X-ray Fracture Screening with Convolutional Neural Networks
The fracture screening feature in KhimiYaar AI represents a sophisticated application of computer vision in healthcare. Using Convolutional Neural Networks (CNNs) implemented in TensorFlow/Keras, the system classifies X-ray images as either “fractured” or “not fractured”.
The model architecture typically involves multiple convolutional layers for feature extraction, followed by pooling layers for dimensionality reduction, and dense layers for final classification. Advanced data augmentation techniques—such as rotation, zoom, and horizontal flipping—enhance model robustness and generalization.
Step-by-Step Guide: Building a Fracture Detection Model
import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.preprocessing.image import ImageDataGenerator
<ol>
<li>Set up data augmentation and generators
train_datagen = ImageDataGenerator(
rescale=1./255,
rotation_range=20,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
validation_split=0.2
)</li>
</ol>
train_generator = train_datagen.flow_from_directory(
'data/train',
target_size=(224, 224),
batch_size=32,
class_mode='binary',
subset='training'
)
validation_generator = train_datagen.flow_from_directory(
'data/train',
target_size=(224, 224),
batch_size=32,
class_mode='binary',
subset='validation'
)
<ol>
<li>Build the CNN model
model = models.Sequential([
layers.Conv2D(32, (3, 3), activation='relu', input_shape=(224, 224, 3)),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(64, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(128, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(128, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
layers.Flatten(),
layers.Dropout(0.5),
layers.Dense(512, activation='relu'),
layers.Dense(1, activation='sigmoid')
])</p></li>
<li><p>Compile and train
model.compile(optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy'])</p></li>
</ol>
<p>history = model.fit(
train_generator,
steps_per_epoch=100,
epochs=50,
validation_data=validation_generator,
validation_steps=50
)
<ol>
<li>Save the model for deployment
model.save('models/fracture_detection.h5')
For improved accuracy, many implementations fine-tune pre-trained architectures like ResNet50, which can achieve approximately 85% accuracy in fracture classification. The model can then be deployed via a Gradio or Streamlit interface for real-time diagnostic predictions.
4. Multilingual Support and Emergency Detection
KhimiYaar AI’s support for both English and Roman Urdu addresses a critical accessibility gap in healthcare information. Implementing multilingual support requires careful consideration of NLP pipelines that can handle code-switching and transliterated text. The platform likely uses language detection models to route queries to the appropriate language model or translation layer.
The emergency detection feature with region-specific contact information adds a crucial safety layer. This involves geolocation services or user-provided location data to display appropriate emergency numbers (e.g., 911 in the US, 112 in Europe, 15 in Pakistan). The system can also analyze user input for distress signals—keywords like “heart attack,” “stroke,” or “severe pain”—and trigger immediate emergency prompts.
Step-by-Step Guide: Implementing Emergency Detection
Emergency detection logic
EMERGENCY_KEYWORDS = ['heart attack', 'stroke', 'severe pain', 'bleeding', 'unconscious']
REGION_EMERGENCY_NUMBERS = {
'US': '911',
'UK': '999',
'Pakistan': '15',
'India': '102',
'Default': '112'
}
def detect_emergency(user_input, region='Default'):
user_input_lower = user_input.lower()
for keyword in EMERGENCY_KEYWORDS:
if keyword in user_input_lower:
return {
'emergency': True,
'keyword': keyword,
'contact': REGION_EMERGENCY_NUMBERS.get(region, '112'),
'message': f"⚠️ Emergency detected! Please call {REGION_EMERGENCY_NUMBERS.get(region, '112')} immediately."
}
return {'emergency': False}
5. Security, Privacy, and Responsible AI in Healthcare
Healthcare applications handling sensitive patient data require robust security measures. KhimiYaar AI must implement encryption for data in transit (TLS/SSL) and at rest (AES-256). API endpoints should be secured with authentication tokens (JWT) and rate limiting to prevent abuse.
Step-by-Step Guide: Securing FastAPI Endpoints
from fastapi import Depends, HTTPException, status
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
import jwt
from datetime import datetime, timedelta
security = HTTPBearer()
Secret key for JWT (store in environment variables)
SECRET_KEY = "your-secret-key-here"
def create_access_token(data: dict, expires_delta: timedelta = None):
to_encode = data.copy()
if expires_delta:
expire = datetime.utcnow() + expires_delta
else:
expire = datetime.utcnow() + timedelta(minutes=15)
to_encode.update({"exp": expire})
return jwt.encode(to_encode, SECRET_KEY, algorithm="HS256")
def verify_token(credentials: HTTPAuthorizationCredentials = Depends(security)):
token = credentials.credentials
try:
payload = jwt.decode(token, SECRET_KEY, algorithms=["HS256"])
return payload
except jwt.ExpiredSignatureError:
raise HTTPException(status_code=401, detail="Token expired")
except jwt.InvalidTokenError:
raise HTTPException(status_code=401, detail="Invalid token")
Protected endpoint
@app.post("/predict/fracture")
async def predict_fracture(
file: UploadFile = File(...),
user: dict = Depends(verify_token)
):
Only authenticated users can access this endpoint
...
Responsible AI practices also include model explainability—providing users with confidence scores and contextual information about why a particular prediction was made. This transparency builds trust and allows users to make informed decisions about seeking professional medical advice.
What Undercode Say:
- Key Takeaway 1: The integration of multiple AI modalities—conversational AI, computer vision, and NLP—into a single platform demonstrates the maturity of modern AI tooling. React, FastAPI, and TensorFlow form a powerful triad that enables rapid development of complex, production-ready AI applications.
-
Key Takeaway 2: Cultural and linguistic accessibility is not an afterthought but a core design principle. By supporting Roman Urdu and incorporating region-specific emergency contacts, KhimiYaar AI addresses real-world barriers that prevent underserved populations from accessing healthcare information.
Analysis:
The project exemplifies how a solo developer or small team can leverage open-source technologies to build impactful AI solutions. The choice of FastAPI over Django or Flask is strategic—its async capabilities and automatic OpenAPI documentation make it ideal for ML serving. TensorFlow’s ecosystem provides the flexibility to experiment with different model architectures, from custom CNNs to fine-tuned ResNet50. React’s component-based architecture enables rapid UI iteration, crucial for a platform that must be intuitive for users of varying technical literacy. The project also highlights the importance of responsible AI: KhimiYaar AI is positioned as a supplement to professional medical advice, not a replacement, with emergency detection serving as a critical safety net. This responsible approach is essential for any AI healthcare application, as misdiagnosis or over-reliance on AI could have serious consequences.
Prediction:
+1: KhimiYaar AI represents a template for how AI can democratize healthcare access in regions with limited medical infrastructure. Similar projects will emerge in other underserved communities, each adapted to local languages and healthcare needs.
+1: The platform’s multi-modal approach—combining text, image, and conversational AI—will become the standard for consumer health applications, moving beyond single-purpose symptom checkers to comprehensive health companions.
-1: Regulatory challenges will intensify as AI healthcare platforms proliferate. Without clear guidelines on liability and data privacy, developers may face legal hurdles that could stifle innovation or force platforms to restrict their features.
-1: The risk of over-reliance on AI for medical decisions remains significant. Even with clear disclaimers, users may trust AI predictions over professional medical advice, potentially leading to delayed treatment or misdiagnosis. Continuous user education and robust emergency detection are essential mitigations.
▶️ Related Video (74% Match):
🎯Let’s Practice For Free:
🎓 Live Courses & Certifications:
Join Undercode Academy for Verified Certifications
🚀 Request a Custom Project:
Secure, high-velocity infrastructure and disruptive technological engineering. Contact our engineering team for high-tier development and proprietary systems:
[email protected]
💎 Smart Architecture | 🛡️ Secure by Design | ⭐ Trusted by Thousands
IT/Security Reporter URL:
Reported By: Aadil Kamal – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅
