Following could be the definitions of the APIs for creating and deleting URLs:Įnter fullscreen mode Exit fullscreen mode We can have SOAP or REST APIs to expose the functionality of our service. One thing to note here is that since there will be many duplicate requests (of the same URL), our actual memory usage will be less than 170GB. To cache 20% of these requests, we will need 170GB of memory: 0.2 * 1.7 billion * 500 bytes = ~170GB Since we have 20K requests per second, we will be getting 1.7 billion requests per day: 20K * 3600 seconds * 24 hours = ~1.7 billion If we want to cache some of the hot URLs that are frequently accessed, how much memory will we need to store them? If we follow the 80–20 rule, meaning 20% of URLs generate 80% of traffic, we would like to cache these 20% hot URLs. We will need 15TB of total storage: 30 billion * 500 bytes = 15 TBįor write requests, since we expect 200 new URLs every second, total incoming data for our service will be 100KB per second: 200 * 500 bytes = 100 KB/sįor read requests, since every second we expect ~20K URLs redirections, total outgoing data for our service would be 10MB per second: 20K * 500 bytes = ~10 MB/s Let’s assume that each stored object will be approximately 500 bytes (just a ballpark estimate–we will dig into it later). Since we expect to have 500M new URLs every month, the total number of objects we expect to store will be 30 billion: 500 million * 5 years * 12 months = 30 billion Let’s assume we store every URL shortening request (and associated shortened link) for 5 years. What would be Queries Per Second (QPS) for our system? New URLs shortenings per second: 500 million / (30 days * 24 hours * 3600 seconds) = ~200 URLs/sĬonsidering 100:1 read/write ratio, URLs redirections per second will be: 100 * 200 URLs/s = 20K/s Let’s assume a 100:1 ratio between read and write.Īssuming, we will have 500M new URL shortenings per month, with 100:1 read/write ratio, we can expect 50B redirections during the same period: 100 * 500M => 50B There will be lots of redirection requests compared to new URL shortenings. Our service should also be accessible through REST APIs by.Analytics e.g., how many times a redirection happened?.Shortened links should not be guessable (not predictable).URL redirection should happen in real-time with minimal latency.This is required because, if our service is down, all the URL redirections will start failing. Such a system should be highly available. Users should be able to specify the expiration time.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |