OCTOBER 14, 2015

Near Engineering

by Madhu Therani

Chief Technology Officer

Welcome to the Tech blog at Near.co. With our ongoing mission of building out the world’s largest location intelligence platform, the tech team at Near is working on a number of bleeding-edge technologies in distributed systems, large-scale data processing pipelines, data storage, and data sciences. The close-knit, world-class team works on a number of interesting areas building in-house tech and adapting open-source systems. Key areas and associated topics of interest for us include:

Distributed Systems

  • Maintaining and updating state across our sharded environment as we receive concurrent events at scale (Cloud infrastructures)
  • Reacting to state changes and key events at different time scales
  • Storing state, Restoring state, Replaying event histories at scale (MongoDB, Cassandra, Redis)
  • Searching across our heterogeneous data repositories (ElasticSearch, Mongo)
  • Integrations with other external systems in real-time, batch and micro-batch modes - capturing and filtering events of relevance

Data Engineering

  • Processing tera-scale events in our Hadoop, Storm and Apache Spark pipelines
  • Managing both realtime event processing and batch-mode data in a unified framework as our data pipelines process different kinds of data - across verticals
  • Distributed coordination across our compute and storage clusters via scalable messaging (Kafka, ZeroMQ)
  • Real-time unstructured data pipelines - NLP, ML and Geo data processing

Data Sciences

  • Data integration across different formats, schemas, sources to support our platform APIs - places, people, products and other entities
  • Analytics across our data infrastructure - (Hive, Pig, SparkSQL and more)
  • Model development - inferring people attributes, place attributes and more
  • Handling Geo-centric data in all its guises

Mobile Apps & Systems

  • Building Apps and SDKs that use “mobile devices” as sensors (Android, IOS)
  • Integrating different wearables and other IOT components

Product Development & Deployment

The data harnessed by the Near platform is utilized in a variety of products and applications. Tasks in product development include:

  • Analytics visualizations
  • UI design and development
  • Scaling devops across our platform and products
  • Realworld product testing and management

We are excited by the potential of our platform and the kinds of applications we can power in the future, especially as a techie. If you are interested  in working with us, please follow our tech blog and also look up our careers page or reach out to us at tech@near.co.