Automation, ETL, Data Pipeline from MongoDB to Kibana, Elasticsearch
Are you seeking a similar solution?
Please reach me:
Email: [email protected]
WhatsApp: +91 9717367468
LinkedIn: linkedin.com/in/asbidyarthy
Automation, ETL, Data Pipeline, MongoDB, Kibana, Elasticsearch, MongoDB to Elasticsearch, MongoDB to Kibana, ETL Pipeline, Data Engineering, Data Integration, Data Migration, ELK Stack, Log Analytics, Big Data, Data Processing, Apache Kafka, Data Automation, Database Management, Data Visualization, Real Time Analytics, Business Intelligence, Cloud Data Pipeline, Python ETL, Elasticsearch Dashboard, Kibana Dashboard, DevOps Automation, Data Analytics, Blackcoffer, Data Engineering Project
#Automation, #ETL, #DataPipeline, #MongoDB, #Kibana, #Elasticsearch, #ELKStack, #DataEngineering, #DataIntegration, #DataMigration, #BigData, #LogAnalytics, #RealTimeAnalytics, #BusinessIntelligence, #DataVisualization, #Python, #ApacheKafka, #CloudComputing, #DevOps, #DataAnalytics, #Dashboard, #TechProjects, #Blackcoffer, #Database, #SoftwareDevelopment
We query documents from mongo using last modified data (for modified documents) and Instance created date (for created documents), comparing them against last run time stored and updated on mongo DB , from section instance and file instance, and merge both of them and push in onto Kibana using bulk api. Also logged the errors on a separate elastic search index for errors.