The customer needs to develop a solution for monitoring and collecting information on various resources to increase its sales.
The developed system should significantly accelerate and automate the routine actions taken to search for and collect contact data of potential customers.
The system is designed to collect and display data from such sources as Crunchbase, Angel.co, etc., Google Play and App Store. All information received is integrated with the customer’s CRM.
The application allows collecting any information that may be useful for the sales department.
The received data is stored in a cloud database. Sorting by different fields, as well as export of data to CSV were implemented. The project is deployed on the Google Cloud infrastructure.
The parsers are run according to the scheduler, so it allows avoiding too high load on the customer’s capacities. In addition to the main crawlers and parsers, there is a queue for the updating parser, which allows updating information on the data that has already been collected. There is also a special service that displays the “health” of the parsers. This makes it possible to prevent serious errors in the system operation in a timely manner and to implement the necessary changes in time to restore operability.