This week’s system design refresher:
Top 3 Things You Should Know About Webhooks (Youtube video)
Cloud Load Balancer Cheat Sheet
How do we manage sensitive data in a system?
What does ACID mean?
SPONSOR US
There’s a great new way to learn the basics of streaming data — enroll in Redpanda University for free, online, self-paced training. Courses include as many as 8 lessons and end with a brief quiz.
Start with the Streaming Data Fundamentals course to get a solid foundation of streaming data knowledge. More advanced courses such as cluster management are available, too!
Efficient load balancing is vital for optimizing the performance and availability of your applications in the cloud.
However, managing load balancers can be overwhelming, given the various types and configuration options available.
In today's multi-cloud landscape, mastering load balancing is essential to ensure seamless user experiences and maximize resource utilization, especially when orchestrating applications across multiple cloud providers. Having the right knowledge is key to overcoming these challenges and achieving consistent, reliable application delivery.
In selecting the appropriate load balancer type, it's essential to consider factors such as application traffic patterns, scalability requirements, and security considerations. By carefully evaluating your specific use case, you can make informed decisions that enhance your cloud infrastructure's efficiency and reliability.
This Cloud Load Balancer cheat sheet would help you in simplifying the decision-making process and helping you implement the most effective load balancing strategy for your cloud-based applications.
Over to you: What factors do you believe are most crucial in choosing the right load balancer type for your applications?
If you’re not a paid subscriber, here’s what you missed this month.
To receive all the full articles and support ByteByteGo, consider subscribing:
The cheat sheet below shows a list of guidelines.
What is Sensitive Data?
Personal Identifiable Information (PII), health information, intellectual property, financial information, education and legal records are all sensitive data.
Most countries have laws and regulations that require the protection of sensitive data. For example, the General Data Protection Regulation (GDPR) in the European Union sets stringent rules for data protection and privacy. Non-compliance with such regulations can result in hefty fines, legal actions, and sanctions against the violating entity.
When we design systems, we need to design for data protection.
Encryption & Key Management
The data transmission needs to be encrypted using SSL. Passwords shouldn’t be stored in plain text.
For key storage, we design different roles including password applicant, password manager and auditor, all holding one piece of the key. We will need all three keys to open a lock.
Data Desensitization
Data desensitization, also known as data anonymization or data sanitization, refers to the process of removing or modifying personal information from a dataset so that individuals cannot be readily identified. This practice is crucial in protecting individuals' privacy and ensuring compliance with data protection laws and regulations. Data desensitization is often used when sharing data externally, such as for research or statistical analysis, or even internally within an organization, to limit access to sensitive information.
Algorithms like GCM store cipher data and keys separately so that hackers are not able to decipher the user data.
Minimal Data Permissions
To protect sensitive data, we should grant minimal permissions to the users. Often we design Role-Based Access Control (RBAC) to restrict access to authorized users based on their roles within an organization. It is a widely used access control mechanism that simplifies the management of user permissions, ensuring that users have access to only the information and resources necessary for their roles.
Data Lifecycle Management
When we develop data products like reports or data feeds, we need to design a process to maintain data quality. Data developers should be granted with necessary permissions during development. After the data is online, they should be revoked from the data access.
The diagram below explains what ACID means in the context of a database transaction.
Atomicity
The writes in a transaction are executed all at once and cannot be broken into smaller parts. If there are faults when executing the transaction, the writes in the transaction are rolled back.
So atomicity means “all or nothing”.
Consistency
Unlike “consistency” in CAP theorem, which means every read receives the most recent write or an error, here consistency means preserving database invariants. Any data written by a transaction must be valid according to all defined rules and maintain the database in a good state.
Isolation
When there are concurrent writes from two different transactions, the two transactions are isolated from each other. The most strict isolation is “serializability”, where each transaction acts like it is the only transaction running in the database. However, this is hard to implement in reality, so we often adopt loser isolation level.
Durability
Data is persisted after a transaction is committed even in a system failure. In a distributed system, this means the data is replicated to some other nodes.
Get your product in front of more than 500,000 tech professionals.
Our newsletter puts your products and services directly in front of an audience that matters - hundreds of thousands of engineering leaders and senior engineers - who have influence over significant tech decisions and big purchases.
Space Fills Up Fast - Reserve Today
Ad spots typically sell out about 4 weeks in advance. To ensure your ad reaches this influential audience, reserve your space now by emailing [email protected]