AI Scraping & LLM Engineering Manager - Python

About Comapny

---

AIMLEAP is an ISO 9001:2015 and ISO/IEC 27001:2013 certified global technology consulting and service provider offering Digital IT, AI-augmented Data Solutions, Automation, and Research & Analytics Services.
AIMLEAP has been recognized as ‘The Great Place to Work®’. With focus on AI and automation-first approach, our services include end-to-end IT application management, Mobile App Development, Data Management, Data Mining Services, Web Data Scraping, Self-serving BI reporting solutions, Digital Marketing, and Analytics solutions.
We started in 2012 and successfully delivered projects in IT & digital transformation, automation driven data solutions, and digital marketing for more than 750 fast-growing companies in the USA, Europe, New Zealand, Australia, Canada; and more.
– An ISO 9001:2015 and ISO/IEC 27001:2013 certified
– Served 750+ customers
– 12+ Years of industry experience
– 98% Client Retention
– Great Place to Work® Certified
– Global Delivery Centers in the USA, Canada, India & Australia.

What We Are Looking For

• 9–15 years of experience in building large-scale web scraping, browser automation, and data extraction systems using Python. 
• Proven expertise in Scrapy, Selenium, Playwright, Puppeteer, and modern browser automation frameworks for enterprise-grade scraping projects. 
• Strong experience in building AI-powered scraping workflows using LLMs for intelligent extraction, data normalization, classification, and automation. 
• Experience in designing scalable distributed scraping pipelines with proxy rotation, anti-bot handling, CAPTCHA solving, and high-concurrency crawling systems. 
• Proven ability to lead engineering teams, manage delivery timelines, conduct code reviews, and drive architecture decisions for AI/data platforms. 

Responsibilities:

• Lead the architecture and development of AI-driven scraping and browser automation platforms for structured and unstructured web data extraction. 
• Build intelligent scraping pipelines leveraging LLMs, LangChain, LlamaIndex, OpenAI, Claude, or open-source AI models for automated extraction and enrichment. 
• Design and manage scalable scraping infrastructure using Scrapy, Selenium, Playwright, Docker, Kubernetes, queues, and cloud-native deployment systems. 
• Collaborate with AI, product, and engineering teams to create next-generation intelligent data acquisition and automation solutions. 
• Mentor and manage scraping/data engineering teams while establishing coding standards, best practices, monitoring, and scalable engineering processes. 
• Optimize system performance, reliability, scalability, and deployment workflows using AWS/GCP, CI/CD pipelines, databases, and distributed processing systems. 

Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Engineering, IT, or related field. 
• Strong hands-on expertise in Python, Scrapy, Selenium, Playwright, FastAPI/Django, APIs, and scalable scraping architectures. 
• Experience working with LLMs, AI automation frameworks, vector databases, semantic search, embeddings, and intelligent extraction systems. 
• Proficiency in Docker, Kubernetes, AWS/GCP, PostgreSQL/MySQL/MongoDB, Redis, CI/CD, and cloud infrastructure management. 
• Prior experience leading engineering teams and delivering enterprise-scale scraping, AI automation, or data intelligence products. 
• Excellent analytical, communication, leadership, and problem-solving skills with experience working in Agile development environments.