This week’s system design refresher:
How does HTTPS work? (Youtube video)
Types of databases and use cases
ElasticSearch
REST API Design
With Ilum's solution, everyone can now quickly and easily deploy Apache Spark on any Kubernetes cluster. Our software eliminates the need for tedious configuration and reduces the time needed for deployment from days to minutes.
By leveraging the power of container orchestration and Apache Spark's scalability and reliability, we are making it easier than ever to stay ahead of the curve and explore the future of Big Data.
Ilum provides an all-in-one solution for:
Apache Spark Cluster management and monitoring
Managed Spark service
Hadoop replacement
Apache Livy alternative
Spark Session control over REST API
Real-time interaction with Spark jobs
And the best part? It's free! Unlock the power of Big Data today with Ilum.
There are hundreds or even thousands of databases available today, such as Oracle, MySQL, MariaDB, SQLite, PostgreSQL, Redis, ClickHouse, MongoDB, S3, Ceph, etc. How do you select the architecture for your system? My short summary is as follows:
Relational database. Almost anything could be solved by them.
In-memory store. Their speed and limited data size make them ideal for fast operations.
Time-series database. Store and manage time-stamped data.
Graph database. It is suitable for complex relationships between unstructured objects.
Document store. They are good for large immutable data.
Wide column store. They are usually used for big data, analytics, reporting, etc., which needs denormalized data.
Based on the Lucene library, Elasticsearch provides search capabilities. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents. The diagram below shows the outline.
Features of ElasticSearch:
Real-time full-text search
Analytics engine
Distributed Lucene
ElasticSearch use cases:
Product search on an eCommerce website
Log analysis
Auto completer, spell checker
Business intelligence analysis
Full-text search on Wikipedia
Full-text search on StackOverflow
The core of ElasticSearch lies in the data structure and indexing. It is important to understand how ES builds the term dictionary using LSM Tree (Log-Strucutured Merge Tree).
👉 Over to you: Have you used ElasticSearch in your project, and what is it for?
What are its principles, methods, constraints, and best practices? I hope the diagram below gives you a quick overview.
Blog post by Love Sharma.
Openedges: Chief Architect (San Jose, Austin, Remote)
Heir: Senior Software Engineer, Full Stack (United States)