Rooftop Corner Detection API

AI-powered FastAPI service that detects rooftops in images and extracts their corner coordinates using YOLO instance segmentation and classic computer-vision post-processing.

Overview

This service accepts an uploaded image, runs a YOLO segmentation model to produce a rooftop mask, refines the mask with morphological smoothing, and then traces the largest rooftop contour to extract an ordered list of corner points. The corners are returned as pixel coordinates over a clean JSON API — useful for downstream measurement, roof outlining, and property/roofing analysis workflows.

An experimental variant (roofpointerwithVLM.py) adds an open-vocabulary detection stage (OWL-ViT) to constrain the segmentation mask to the rooftop region using natural-language prompts before corner extraction.

Features

• Rooftop segmentation with a trained YOLO segmentation model (updated-seg-v2.pt).
• Corner coordinate extraction via contour detection (findContours) and polygon approximation (approxPolyDP), with downsampling to a manageable corner count.
• Mask refinement using morphological closing to smooth segmentation edges.
• JSON API returning corner count and ordered (x, y) corner coordinates.
• CORS enabled for browser/front-end integration.
• CUDA support with automatic CPU fallback.
• Optional VLM refinement (OWL-ViT open-vocabulary detection) to isolate the rooftop and exclude trees, vegetation, pools, and driveways.

Tech Stack

• Python 3.11
• FastAPI + Uvicorn (API server)
• Ultralytics YOLO (instance segmentation)
• OpenCV (mask processing, contour and corner detection)
• PyTorch / TorchVision (inference backend)
• Transformers (OWL-ViT) (optional open-vocabulary refinement)
• NumPy, Matplotlib

Setup / Run

1. Install dependencies

   pip install -r requirements.txt

2. Run the application

   uvicorn main:app --reload

3. Access the API

   • API root: http://localhost:8000
   • Interactive docs: http://localhost:8000/docs

API Endpoints

Method	Endpoint	Description
POST	/segment-rooftop/	Upload an image; returns rooftop corner coordinates.
GET	/	Health/status check.

Example response

{
  "status": "success",
  "corners_count": 6,
  "corners": [
    { "index": 0, "x": 412, "y": 188 },
    { "index": 1, "x": 520, "y": 240 }
  ]
}

Deployment (Render)

1. Push to GitHub.
2. Connect the repo to Render.
3. Build command: pip install -r requirements.txt
4. Start command: uvicorn main:app --host 0.0.0.0 --port $PORT

Requirements

• Python 3.11+
• YOLO segmentation weights (updated-seg-v2.pt)

Author

Usama Bin Ejaz — AI/ML Engineer | Data Scientist | Founder, UBE Labs

• GitHub: github.com/ube09
• Website: usamabinejaz.com (UBE Labs — "AI that ships.")