Mahesh Mallikarjunaiah ↗️’s Post

12 Steps to Optimize Software Delivery Cloud Infrastructure: For organizations to be successful and set higher standards for delivery, they will need to think beyond buying infrastructure from cloud provider Code Maintainability: Teams should focus on the importance of managing the code where it is easy to find, reuse, and change across an organization's code repository CICD : Continuous Integration (CI) & Delivery is a software development practice where code changes are automatically built, tested, and integrated into a shared repository multiple times a day. Continuous delivery is where we can release the changes safely. Organizations should follow practices that can drive an effective CICD process. Test Automation: Focusing on building automated tests is essential for organizations to drive delivery optimization and performance Loosely coupled architecture: Evaluating and deciding which architecture patterns fit the best where changes are possible when the need arises Monitoring & Observability: It's essential to build a system that is easy to monitor and continue to debug further Transformational leadership: Having a good leadership team is very essential for the organization Customer feedback: Focusing on the customer feedback and success is vital to drive the product success and in the right direction Test Data Management: Managing test data as code Or as a service is very critical to drive product success Trunk-Based Development: Trunk-based goes well with continuous integration and feature-based releases. Organizations should explore which one suits their needs and adopt it. Visual management: Have a visual map or dashboard where teams can visualize everything from backlog, to processes to know about any critical information for the organization to succeed Security Shift left: Ensuring security testing is emphasised from the beginning of the SDLC is vital for the product to succeed.

"Cloud Native Landscape: Watch Out for These Rising Stars!" These tools can be game changers someday in the areas below. ✔️ 𝗣𝗿𝗼𝘃𝗶𝘀𝗶𝗼𝗻𝗶𝗻𝗴 ✔️𝗥𝘂𝗻𝘁𝗶𝗺𝗲 ✔️𝗢𝗿𝗰𝗵𝗲𝘀𝘁𝗿𝗮𝘁𝗶𝗼𝗻 & 𝗠𝗮𝗻𝗮𝗴𝗲𝗺𝗲𝗻𝘁 ✔️𝗔𝗽𝗽 𝗗𝗲𝗳𝗶𝗻𝗶𝘁𝗶𝗼𝗻 𝗮𝗻𝗱 𝗗𝗲𝘃𝗲𝗹𝗼𝗽𝗺𝗲𝗻𝘁 ✔️𝗢𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗮𝗻𝗱 𝗔𝗻𝗮𝗹𝘆𝘀𝗶𝘀 Do leave a comment if you have used any of these. Reference: CNCF

What happens under the hood when you turn on your Android mobile phone When you power on your mobile phone, a meticulous self-test is conducted on all the hardware components. This comprehensive check ensures that every part of your device is functioning flawlessly. Once this rigorous hardware test is successfully completed, the Boot ROM, stored on the CPU, initiates the loading process. The bootloader loads Android OS, validating certain aspects of the initial loading. The kernel loads device drivers and background processes and manages the system resources. The init process loads and mounts directories and files and starts various processes. The Android runtime is initialized and is mainly used by the Android operating system. As the runtime, ART executes the Dalvik executable (DEX) format and DEX bytecode specification. After these system services are started, the activity manager kicks in to bring the user the initial screen to start interacting. Power-On Self-Test (POST): When you press the power button, the phone undergoes a quick self-check process known as POST. This test ensures that essential hardware components like the processor, memory, and battery function correctly. Bootloader: After the POST is successfully completed, the bootloader, a small yet crucial program stored in the phone's read-only memory (ROM), takes charge. It not only initializes the hardware but also orchestrates the booting process, including selecting the operating system to load. Loading the Kernel: The bootloader loads the Android kernel into the device's RAM (Random Access Memory). The kernel is the core of the Android operating system. It manages the device's resources, such as the CPU, memory, and input/output operations. Init Process: Once the kernel is loaded, the init process (short for initialization) is started. This process is responsible for initializing the Android system and launching essential system services and daemons. Android Runtime (ART) Initialization. This is a critical step as ART is the runtime environment where Android applications come to life. It's responsible for a significant task and compiles the application code into machine code, ensuring efficient execution on the device's hardware. This makes ART a crucial part of the booting process. Starting System Services: After the initialization of ART, various system services are started. These services include the Android System Server, which manages core system components like the package manager, activity manager, and window manager. Launcher and User Interface: Once the system services are up and running, the launcher (the user interface that displays your home screen, app icons, and widgets) is started. This allows you to interact with your device and access your installed applications. Loading Applications: Any pre-installed applications and services configured to start automatically are loaded. These may include system apps like the dialer, messaging app.

Pub/Sub and queue look the same when you hear it for the first time.😱 Here is a simple explanation of what they are and the differences between them. In the image depicted below, 🔴 Messaging Queue: People come to the mall to shop. Before they check out, they must wait in the queue to complete the payment. Here, the mall is like a producer, producing people ( who have shopped and are ready to complete payment) who are sent to the queue to complete the payment. One by one, each person completes payment at the checkout counter. The checkout counter is like a consumer. As the queue grows ( more people join the line), more than one checkout ( one consumer ) is needed, and we will need to add more checkout counters ( more consumers). We have Producers, Queues, and Consumers. 🔴 Pub / Sub - Extending on the above example, after the checkout is completed, this customer data might be required by various services. For instance, in the image below, we have a real-time service that calculates how many people have completed the checkout. Hence, the checkout service has subscribed to receive this data. Likewise, marketing and inventory services might subscribe to receive this data. We have Producers, Topics, and Subscribers. #engineering #systemdesign #interview #productdevelopment #eventdrivenarchitecture

Visualizing the Scrum Framework Scrum is an agile approach to developing and releasing software products Scrum starts with three key roles namely - Product Owner, Scrum Master, and Scrum team / Feature team / Development team The creation of requirements and the determination of development timing and delivery are undertaken by the Product Owner, who is also tasked with maintaining the overall vision of the product. The Scrum master works with the scrum team to help the team understand the importance of scrum principles, values, and processes. They coach the team, along with acting as a facilitator. A critical responsibility of the scrum master is to ensure the team is not blocked with anything and protect the team from any interferences that might impact the delivery. The scrum / Development team comprises folks with cross-functional, diverse skill sets that can help release the agreed-upon sprint goals. The whole team works as one to deliver value The Journey starts with the Product owner writing down the high-level requirements as the product backlog, which are broken down further into small user stories and tasks based on the discussion with the scrum team. This process is also called backlog grooming, which is prioritized. A sprint begins with sprint planning, where items from the backlog are pulled into the sprint backlog based on priority and discussion with the product owner. The user stories are estimated based on various estimate strategies like planning poker and others. The team as a whole agrees on what will be delivered as part of the sprint. Once spirit is started, the team pulls user stories that will be worked upon daily and updates progress. Daily scrum meetings would be conducted to talk about the progress that is being made and also discuss any roadblocks. The user stories should meet the defination of done defined which is very vital. At the end of the sprint a potential shippable product feature will be ready which will be demoed to all stakeholders which might include customers as well. A Sprint retrospective is an activity to inspect what is working, what is not working, what went fine and did not go fine, and discuss and brainstorm, which can lead to embracing and applying changes as necessary for the betterment of the upcoming sprint. The Scrum team invests in wirting Good user stories that should meet the INVEST criteria Various metrics like burnup, burndown, and sprint velocity are monitored to track the progress of the sprint. #scrummaster #agile #scrum #development

YouTube is one of the most popular applications that stream videos. Billions of users worldwide use YouTube, which demands high availability, performance, scalability, reliability, resiliency, etc. Storage and bandwidth estimation are high for YouTube, which means YouTube would require more servers and ensure videos consume less bandwidth without impacting speed, performance, and clarity. When an end user tries to access youtube, it intially uses its Google cloud DNS to divert the users appropriately, which intially hits the webserver ( currently youtube uses Google ESF ). Load balancers are used to distribute the load across servers. The requests are routed to the individual microservices using API gateway services. Segregating web servers and application servers and running them as individual microservices will help with scalability, resiliency, and others When a user tries to upload a video, it uses the upload microservices, which uploads the video to a temproary storage, encoder services picks up the video to convert to various formats, resolutions which could be played back on multiple devices. Also, in parallel, the video metadata is stored in the database. The converted video is stored on object storage like Google Cloud storage. The most viewed vidoes are stored in the CDN, edge location cache closer to the end user location, which can improvise the overall performance and experience of watching vidoes. Caching is implemented across various levels of architecture, including web and app servers. A distributed cache is critical to ensure no single point of failure Thumbnails are generated and stored Google File System ISP provider's colocation sites are used for storing some of the famous videos for faster delivery NoSQL and SQL databases are combined; NoSQL DB has metadata stored. Sharding and replication are adopted to help with the growth, scalability, and disaster recovery scenarios. Using the keywords and ranking algorithm, the appropriate vidoes are fetched A robust observability system is in place to ensure the system meets the SLAs and provides service continuity. According to the CAP theorm, high availability means strong consistency will take a hit, but for YouTube, eventual consistency will be fine since slight variation for a temporary duration on total likes or comments will not have a considerable imapct YouTube recommendation engine uses machine learning to provide video recommendations to its users based on their profile, likings, content ranking, and other factors. The adaptive streaming algorithm is adopted to serve videos to users based on their bandwidth. Depending on the speed, the quality of the chunk is varied. Vitess is used to overcome the challenges of MySQL scaling across multiple servers. It adopts a shard routing algorithm and encapsulates everything so app servers don't feel the overhead. #systemdesign #engineering #productdevelopment #development

Amazon Prime has a metric of 99.999%, which translates to an unplanned downtime of 26 seconds per month. How Prime Video ingests, processes, and distributes live TV to millions Amazon Prime delivers live streaming to millions of customers across the globe. The mean time between failures is the average time between failures. MBTF combined across all the components provides the overall availability Components in Series = Availability of x ∗ availability of y Components in Parallel = Availability=1–(1–Availability of x)∗(1–Availability of y) Since AWS provides an SLA of 99.9% for each component Overall SLA of components in Series = 99.7% Overall SLA of components in Parallel = 99.999% Running in Parallel affects the SLA's Video is delivered in a sequence of packets. The breakdown is as follows. Camera - Video Processing System - Packets - Frames - Pictures (GOPs) - HTTP - Customer Device If the above sequence is broken, it causes video buffering. Prime Video observes/monitors all components, including broadcaster sources, to manage any issues. Prime chooses the AWS region closest to both signal source and customer to optimize for cost and deliver quality. It uses AWS Direct Connect, a global accelerator when there isn't a local region available. To keep up with the SLA, prime avoids building a system with a single point of failure. It deployed in two AWS regions as active-active Partners deliver two feeds to each AWS elemental mediaconnect so that if one fails, the backup can be activated, and that way, the video pipeline isn't disrupted and this can lead to just 1s of picture degradation Prime also reduces the number of bits delivered to improve quality and minimize buffering. They also optimize Adaptive bitrate. They have a quality standards set that partners need to adhere to Prime video is secured across the pipeline from camera to customer. 1. The integrity of the content 2. Content accessed by authorized users only 3. Content is not exposed externally 4. Content is resilient to piracy 5. Subscribed customers watch the content Prime also uses low-power hardware, reducing power and cooling requirements that drive AWS's sustainability pillar.

Types of data There are various types of data one needs to understand So that one can plan well regarding data capture, management, and usage. Below are some of them:- I have purposely left one box missing, Let me know what data type you feel is important. Structured data - Typically in some form of structure, stored in a tabular form. RDBMS databases like Oracle Geospatial data - Data Related to locations on earth Streaming Data - The output of a lot of data in large volumes continuously, like gaming, videos etc Synthetic Data - Tools or AI technology artificially create this data, which is utilized for testing, research, and machine learning. Binary Data - Data which is usually stored in the format of audio, video files Sentiment Data - Data like emails, chats, and comments are used to analyze the emotional tone of the message Spatiotemporal data - This data includes location and time, which enables tracking, like cab tracking Open data - The data that is free for the public to use, edit, and share for any purposes Real-time Data - With 5g technology, this data is captured in real-time and analyzed in real-time, which is valuable for traffic emergency systems. High-dimensional data - The number of variables (or dimensions) is significantly larger than the number of samples, for example, used in IOT smart farming and facial recognition. Transactional Data - Data generated due to various transactions like payment, e-commerce shopping, Batch Data - Data in high volumes is collected and processed Dark Data - Information that is collected and stored but not used in any meaningful way Machine Data - Data generated by the systems automatically across an organization's data center Genomics Data - Data about the organization and operation of an organism's genetic material. For example, analyzing DNA of patients Big Data - Large and complex data sets that doesn't fit the traditional RDBMS Biometric - Data that deals with biometic data like face, finger prints and voice Sensor Data - Data collected from various sensors like temperature, Gyroscope sensors Social Media Data - Data collected from all the social media applications and further leveraged for various purposed Semi-Structured - Data which is in a format like JSON and which doesn't conform to data model Unstructured Data - Text files in PDF, email that doesn't follow any structure or data model Time series data - Data captured over interval of time continuously and consistently #data #dataanalytics #engineering #machinelearning

🚀 The Test Tribe's 35th Bangalore Meetup is happening tomorrow, and it's going to be an event you won't want to miss! 🎉 Join us for insightful sessions led by industry experts Gaurav Singh and Harish Pandalangatt. 🎙️ Get ready to explore the latest trends, best practices, and cutting-edge technologies in the testing world. 👉🏻 Register from here: https://bit.ly/ttt35blr Shashank R. 🚀 #TTT35BLR #TechMeetup #Networking #LearningAndFun #ttt