Lead Software Engineer - Scraping job at Next Ventures Ltd New York, NY, US

Job Description

Role:

Our client is an innovative FinTech company and the market leader in providing data and business intelligence to help customers make faster and smarter investment decisions. They are looking for a Lead Software Engineer to drive and own theit web scraping initiatives. This role will lead the design, development, and maintence of scalable scraping solutions as part of their data technology team. The ideal candidate is an expert Python developer with strong experience in web scraping and related technologies.

Responsibilities

Provide effective technical leadership and hands-on technical mentorship to developers.
Lead the design and development of web scraping applications and services, ensuring they are scalable, efficient, and meet performance requirements.
Drive technical decisions, ensuring that best practices are followed.
Provide leadership in applying design patterns and adhering to SOLID engineering principles, with a focus on reusability, modularity, scalability, and fault tolerance.
Utilize hands-on experience to design, build, and support web scraping solutions, particularly using Python and relevant libraries (e.g., Scrapy).
Promote a culture of continuous improvement and high-quality code delivery.
Partner with the broader technology leadership team to ensure service reliability through testing and release procedures.
Stay current with industry trends in web scraping, data technologies, and emerging tools.
Identify opportunities for innovation and process improvement,
Ensure compliance with security and firmwide architecture standards.

Requirements

Strong understanding of technical design patterns and SOLID engineering principles.
Experience mentoring engineering teams, particularly in the context of data extraction and scraping projects.
Expertise in Python development, particularly with web scraping libraries like Scrapy/Beautiful Soup.
Proven experience in developing and maintaining large-scale web scraping solutions.
Familiarity with C#/.NET technologies is a strong plus.
Strong problem-solving skills and a proactive attitude.
Strong understanding of API design principles and best practices
Experience with CI/CD pipelines and automated testing frameworks.
Familiarity with cloud platforms (AWS/GCP/Azure)
Experience with software development lifecycle tools such as JIRA, Confluence, GitHub, etc.
Excellent communication and collaboration skills.

This position offers comprehensive benefits and is eligible for an annual discretiory bonus.

Fully remote candidates will be considered.

Cannot provide Visa sponsorship at this time.

Job Tags

Permanent employment, Remote job,

Similar Jobs

Knightbridge Capital

Operations Manager Job at Knightbridge Capital

...The Role We are seeking a self-starter, highly organized Operations Manager to support a fast-paced, entrepreneurial commercial real... ...support with day-to-day operational oversight, HR coordination, legal/transactional assistance, investor relations workflows, marketing...

Great American Insurance Group

Senior P&C Underwriter Job at Great American Insurance Group

...Be Here. Be Great. Working for a leader in the insurance industry means opportunity for you. Great American Insurance Group's member companies are subsidiaries of American Financial Group. We combine a "small company" culture where your ideas will be heard with "big company...

Dignity Health

Pharmacy Tech Job at Dignity Health

**Responsibilities**As a Pharmacy Technician at Community Hospital of San Bernardino, you will support the pharmacy team in preparing and dispensing medications, IV admixtures, and chemotherapy while ensuring accuracy and compliance with hospital policies. Your role is...

3SS

AT QA Engineer Job at 3SS

...We are looking for a professional AT Quality Assurance Engineer (AT QA Engineer). Together with... ...Knowledge about iOS and Android OS Working independent Git, GitLab Understanding... ...you look up to now Work From Home Paid Time Off Private Health Insurance...

Professional Search Network

Event Coordinator Job at Professional Search Network

...Primary Duties and Responsibilities: Coordination of onsite and virtual meetings Daily phone and email support up to and including communication with sales representatives and faculty Run various report to manage workload effectively and efficiently For...