Are you passionate about building a platform that leverages crowd? Do you love working with an energetic and cross-functional group of people? Does pioneering innovative ways to understand and interact with LLMs sound exciting to you? Are you passionate about quality of the products and want to contribute to product improvement driven by insights from crowd and LLM? If so, Crowd Intelligence Platform team might be an excellent place for you to grow your career.

The Crowd Intelligence Platform team within Bing Fundamentals is the central team that evaluates the quality of products within Microsoft AI leveraging crowdsourcing and LLM’s. We have an ambition to scale to all products in Microsoft, providing necessary infrastructure to schedule & distribute a large number of product scenarios for validation. This system leverages the power of crowdsourcing to auto generate a user perceived quality score representative of the health of functionalities offered by the product, to enable teams to be in control of their feature quality and take pride in their product.

The service is expected to run 10s of thousands of scenarios every day for both webpages and apps starting with deployment of the right version of operating system/hosting app with appropriate settings and connecting with relevant judges across multiple crowdsourced platforms. In addition, the service will include capabilities to capture the validation process for future debuggability, built in intelligence to target high fidelity judges, ability to dynamically update the pricing for these tasks and support real time routing across platforms to optimize for cost. To name a few products this service is expected to support - Bing, Maps, Edge, Windows Feeds.

We are seeking a Senior Software Engineer to join our Crowd Intelligence Platform team in Microsoft AI. A suitable candidate is expected to have experience of designing and delivering reliable & scalable distributed services. Understanding of crowdsourced platforms and or prompt engineering experience will be a plus. The role requires quick prototyping, narrowing down design options to deliver on the vision in collaboration with a strong team of software engineers.

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Responsibilities

• Work with PM team to create MVP scope.

• Design the system at scale for scenarios and canvases to be supported.

• Design Engineering systems that leverage LLMs in a performant manner.

• Collaborate closely with partner teams in Crowdsourcing and other dependencies.

• Implement some core components and manage deployment, live site

• Iterate the service capabilities as new requirements emerge with service rolled out.

• Act as a Tech lead for the area.

Qualifications

Required Qualifications: * *

• Bachelor's Degree in Computer Science, or related technical discipline AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python

• OR equivalent experience.

• 3+ years of experience of designing, coding, debugging and delivering software projects

• Experience with designing and building reliable, distributed services

Preferred Qualifications:

• Master's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python

• OR Bachelor's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python

• OR equivalent experience.

• Demonstrated problem solving and prioritization skills

• Experience with Search products, Telemetry, Metrics.

• Experience of using LLMs and ML in engineering systems will be preferred.

• Knowledge of Crowdsourced platforms.

Software Engineering IC4 - The typical base pay range for this role across the U.S. is USD $117,200 - $229,200 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $153,600 - $250,200 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay

Microsoft will accept applications for the role until November 17, 2024.

Microsoft is an equal opportunity employer. Consistent with applicable law, all qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations (https://careers.microsoft.com/v2/global/en/accessibility.html) .