OpenSearch as SIEM: Halodoc’s Path to Scalable Security Intelligence

As organizations progressively embrace cloud technologies and expand their digital services, the volume and diversity of logs produced by infrastructure, applications, and SaaS platforms are increasing swiftly. For security teams, consolidating these logs, analyzing them efficiently, and reacting to alerts in real time can present significant challenges. This situation may result in visibility gaps, delayed threat detection, and heightened risk, particularly in rapidly evolving sectors like health tech.

At Halodoc, we needed a SIEM solution that was scalable, flexible, and cloud-native to meet our compliance, threat detection, and forensic analysis needs. In this blog, we’ll share how Halodoc uses OpenSearch to streamline our SIEM workflows, from collecting and parsing logs to triaging alerts and building dashboards.

What Is SIEM & Why Do Organizations Need It?

Security Information and Event Management (SIEM) is a consolidated system that gathers, processes, and examines data related to security such as logs generated by servers, applications, cloud services, network devices, and endpoints. By integrating logs and events from various parts of the environment, SIEM allows security teams to observe activities, identify threats, and respond in a coordinated and effective way.

Importance of SIEM:  

1. Real-Time Threat Detection : SIEM helps detect threats as they happen. It monitors for signs like repeated failed logins, suspicious privilege changes, or access from known malicious IPs. When something unusual is found, it instantly sends an alert. This quick response helps stop attacks early and limits potential damage.

2. Centralized Visibility : Instead of checking logs across different systems, SIEM brings all security data into one place. This centralized view makes it easier for security teams to connect the dots between events. It’s especially helpful in spotting complex attacks that try to move between systems unnoticed.

3. Compliance and Audit Readiness : SIEM helps organizations meet regulatory requirements like HIPAA, PCI-DSS, and ISO 27001. It securely stores logs, tracks user activity, and creates detailed audit reports. This simplifies compliance reporting and ensures you're always ready for audits.

4. Strategic Risk Reduction : Modern SIEM tools do more than just detect threats. They support threat hunting, speed up incident response, and reduce alert fatigue by focusing on high-risk issues. This makes your overall security approach stronger and more efficient.

How OpenSearch Became Our Preferred SIEM Platform

At Halodoc, as our infrastructure grew and security needs evolved, we realized the need for a strong, scalable SIEM solution. We wanted a platform that fits well with our cloud-native setup, helps us detect threats early, and stays cost-effective.

Here’s how OpenSearch stood out as the right choice for our evolving SIEM needs:

1. Scattered Log Visibility
Many commercial SIEMs failed to ingest logs directly from S3 or had limitations parsing custom JSON formats. At Halodoc, our logs from AWS services are dumped into a centralized S3 bucket. OpenSearch integrated seamlessly with these, unlike other platforms that required paid ingestion connectors or custom transformation pipelines.

2. Behavioral Anomaly Detection (User and Entity Behavior Analytics - UEBA)
Many tools charge extra for UEBA features. With OpenSearch, we used its built-in anomaly detection to catch unusual behavior like repeated 4xx/5xx errors from the same IP. This helped us flag suspicious activity without extra cost.

3. Cost-Effective and Open Source
Most commercial SIEMs charge based on how much data you send. OpenSearch is open source, so we avoided high licensing costs while still getting advanced features like dashboards, search, and detection.

4. Customization and Flexibility
Some platforms are too rigid. OpenSearch gave us full control to build custom detection rules, create dashboards for specific teams, and send alerts to tools like Jira and Google Chat. This made it easier for our teams to act quickly.

5. Operational Impact at Halodoc
OpenSearch gave our security team real-time visibility. In one case, anomaly detection flagged login attempts from a service account during non-operational hours and from an unusual geolocation, helping us detect potential compromise early. In another, detection rules surfaced unusual Google Drive downloads. These automated alerts, which previously required manual effort, now help us respond faster and reduce risk.

Designing the OpenSearch SIEM Architecture at Halodoc

To strengthen our threat detection capabilities across Halodoc’s rapidly evolving infrastructure, we built a centralized log analytics and monitoring pipeline using OpenSearch. This system was designed to ingest logs from a wide range of sources AWS services, on-prem infrastructure, and third-party tools and transform them into actionable intelligence in near real-time. Our goal was to provide our security team with unified visibility, scalable monitoring, and reliable alerts without sacrificing system performance or increasing operational complexity.

By standardizing logs through a common schema and automating key processes such as parsing, detection, and alerting, OpenSearch now plays a critical role in how we detect and respond to security threats.

1. Log Collection & Centralization
Logs from AWS services (like GuardDuty, CloudTrail), on-prem servers, and third-party tools are sent to a central S3 bucket. AWS logs are delivered via Kinesis Firehose or directly from services. Other systems use agents like FluentD to send logs.

2. Parsing & Standardization
When new logs arrive in S3, a Lambda function is triggered. It converts raw logs into a structured format (e.g., Elastic Common Schema). This ensures consistency regardless of whether the log is from a web server, cloud service, or application.

3. Indexing & Analytics with OpenSearch
Parsed logs are stored and indexed in OpenSearch, allowing fast searches and real-time dashboards. Logs are automatically organized by lifecycle (hot, warm, cold) and deleted after a set time (e.g., 365 days) to reduce costs.

4. Detection Rules & Monitoring
Using the Security Analytics plugin, our team defines detection rules for threats like brute-force logins or suspicious activity. These rules run continuously and trigger alerts when matched.

Here is a sample detection rule to detect file downloads from Google Workspace that contain Indonesian identity information, such as KTP (Kartu Tanda Penduduk):

5. Alerting & Visualization
When a rule is triggered, OpenSearch sends alerts via SNS or email to the SOC. Security team or SOC team can also use OpenSearch Dashboards to explore incidents, monitor trends, and investigate alerts visually.

Understanding Security Analytics and Threat Detection with the help of Use Cases

1. Unusual Activity: Massive File Download on Google Drive
Google Drive is essential for collaboration but poses a risk for data exfiltration, especially from insiders or compromised accounts. Large or unusual downloads can go unnoticed without monitoring. To reduce this risk, Halodoc’s security team actively tracks such activities as part of our data protection strategy.

Log Investigation (via OpenSearch):
We regularly collect Google Drive audit logs into our OpenSearch-based SIEM system. These logs provide important details like:

• User email and identity
• File names and download sizes
• Timestamps
• IP address and device information

To spot unusual activity, we set thresholds based on what’s normal for each user or team like the number of files, total size, and time of access.

Post Investigation:
The alert is sent to the SOC team through Google Chat or SNS. The team checks logs in OpenSearch to review details like user email, file size, time, and IP address. They compare the activity with the user's usual behavior, such as timing and download size. If it's unclear, they may ask the user for more info. If the download is valid, they close the alert. If not, they take further steps like blocking access or starting an investigation (escalation to security team).

2. Data Exposure: Publicly Accessible S3 Bucket (AWS Security Hub Alert)
Amazon S3 buckets are widely used to store business data, but if misconfigured, they may be unintentionally exposed to the internet. This can lead to sensitive files being publicly accessible. To prevent this, we actively monitor security alerts from AWS Security Hub, which consolidates findings from services like S3, GuardDuty, Config, and more.

Log Investigation (via OpenSearch):
When a potential exposure alert is triggered, the security team navigates to OpenSearch (Discover or Dashboard view). These logs include:

• AWS Account ID
• User ID
• Misconfigured action details
• Generator ID (to cross-check with Security Hub finding)

Using this info, we go to the AWS Console to confirm:

• If the bucket is truly public
• Who initiated the action
• Whether it was intentional or a misconfiguration

Post Investigation:
The team checks OpenSearch logs and the AWS Console to confirm if the S3 bucket is truly public. They identify who made the change and whether it was intentional. If it was a mistake, they fix the permissions and inform the responsible team. If it was approved access, they document it. The alert is marked as resolved and rules are updated if needed.

Here is the simple Alert Response Flow that is followed by the SOC team:

Conclusion

At Halodoc, OpenSearch has become a key part of how we protect our systems and data. OpenSearch has emerged as a reliable and scalable choice for building a modern SIEM. It helps security teams centralize logs, detect threats in real time, and respond faster with full control over rules and dashboards. Its open-source nature makes it cost-effective, while still offering advanced capabilities like custom detection logic and real-time alerting. For organizations seeking flexibility, performance, and cost-efficiency, OpenSearch offers a solid foundation for effective security operations.

References

https://opensearch.org/docs/latest/
https://github.com/opensearch-project/OpenSearch
https://docs.aws.amazon.com/opensearch-service/latest/developerguide/

Join us

Scalability, reliability and maintainability are the three pillars that govern what we build at Halodoc Tech. We are actively looking for engineers at all levels and if solving hard problems with challenging requirements is your forte, please reach out to us with your resumé at careers.india@halodoc.com.

Bug Bounty

Got what it takes to hack? Feel free to report a vulnerability in our assets and get yourself a reward through our bug bounty program. Find more details about policy and guidelines at https://www.halodoc.com/security 

About Halodoc

Halodoc is the number 1 all around Healthcare application in Indonesia. Our mission is to simplify and bring quality healthcare across Indonesia, from Sabang to Merauke. We connect 20,000+ doctors with patients in need through our Tele-consultation service. We partner with 3500+ pharmacies in 100+ cities to bring medicine to your doorstep. We've also partnered with Indonesia's largest lab provider to provide lab home services, and to top it off we have recently launched a premium appointment service that partners with 500+ hospitals that allow patients to book a doctor appointment inside our application. We are extremely fortunate to be trusted by our investors, such as the Bill & Melinda Gates Foundation, Singtel, UOB Ventures, Allianz, GoJek, Astra, Temasek, and many more. We recently closed our Series D round and In total have raised around USD$100+ million for our mission. Our team works tirelessly to make sure that we create the best healthcare solution personalised for all of our patient's needs, and are continuously on a path to simplify healthcare for Indonesia.