✨ This guide provides detailed explanations for each compliance area and key requirement in the Application Compliance Evaluation Tracker. It is designed to help evaluators accurately assess software applications.

How to Use This Guide

1. Preparation: Gather all available documentation from the vendor before beginning the evaluation.
2. Systematic Review: Work through each section in order of criticality (I through IV).
3. Evidence Collection: For each requirement, document the specific evidence that supports your compliance assessment.
4. Gap Analysis: Identify areas where the application falls short and determine if these can be mitigated.
5. Decision Making: Use the completed evaluation to make an informed decision about implementing the application.
6. Documentation: Keep all evaluation materials for future reference and compliance audits.

Remember that this evaluation process is not just a checkbox exercise but a risk management activity. The goal is to understand the risks associated with the application and make informed decisions about whether those risks are acceptable given the business benefits.

Tracker File

Use the document at the bottom of this page to collect evidence and assess the risk. Use one tracker per application 🙌🏼

Systematic Review

Section I: Regulatory and Legal Compliance (Highest Criticality)

GDPR Compliance

Data Processing Agreements in Place

What this means: The application vendor must have a formal, written agreement that specifies how they will process personal data on behalf of your company.
What to look for:

• A Data Processing Agreement (DPA) document from the vendor
• Terms that clearly outline roles and responsibilities regarding data processing
• Specifications for how data is handled, stored, and secured
• Provisions for reporting data breaches
• Procedures for handling data subject requests

Example: Microsoft Office 365 provides a comprehensive DPA that details their role as a data processor, security measures implemented, subprocessor management, and breach notification procedures. The DPA explicitly states that Microsoft will assist with data subject requests and return or delete data upon contract termination.

Lawful Basis for Processing Established

What this means: There must be a legitimate legal reason for the application to collect and process personal data.
What to look for:

• Documentation that specifies which lawful basis applies (consent, contract, legal obligation, vital interests, public task, or legitimate interests)
• If using consent, verification that the application properly obtains and records consent
• Confirmation that data is only used for the purpose for which it was collected

Example: Salesforce documents that it processes customer data primarily under the "contract" lawful basis (to fulfill the service contract with your company), while it processes end-user data based on your company's instructions as the data controller. Their privacy documentation should clearly distinguish between different data types and the corresponding lawful bases.

Data Subject Rights Management

What this means: The application must support the rights of individuals to access, rectify, erase, restrict processing, data portability, and object to processing of their personal data.
What to look for:

• Features that allow you to respond to data subject requests
• Ability to export a user's data in a common format
• Functionality to delete or anonymize user data
• Tools to restrict processing of specific user data

Example: HubSpot provides a "Privacy Center" feature that enables administrators to respond to data subject requests by searching for contacts, viewing all their stored data, exporting it in CSV or JSON format, and permanently deleting it when required. The platform also allows for suppression of processing without full deletion.

Data Protection

Encryption Standards (at Rest)

What this means: Data stored by the application must be protected using encryption when it's not being actively used.
What to look for:

• Documentation specifying encryption algorithms used (AES-256 is commonly accepted)
• Encryption of databases and file storage
• Secure key management practices
• Evidence that encryption is actually implemented, not just planned

Example: AWS RDS for MySQL provides transparent data encryption using AES-256. In their documentation, they specify that database files, automated backups, and snapshots are all encrypted. They detail their key management through AWS KMS (Key Management Service) and provide audit capabilities to verify encryption is active.

Encryption Standards (in Transit)

What this means: Data must be protected when it's being transmitted between systems or users.
What to look for:

• Use of HTTPS/TLS for web interfaces (minimum TLS 1.2)
• Secure API communications with encryption
• Proper certificate management
• No support for deprecated, insecure protocols

Example: Slack enforces TLS 1.2+ for all web, mobile, and API communications. Their security documentation specifies they use ECDHE cipher suites for perfect forward secrecy and maintain an A+ rating on SSL Labs tests. They should provide documentation showing how certificates are managed and rotated.

Security Certifications

What this means: The application or vendor has undergone independent verification of their security practices.
What to look for:

• SOC 2 Type II certification
• ISO 27001 certification
• Other relevant industry certifications
• Verification that certifications are current and cover the specific application

Example: Dropbox Business provides its SOC 2 Type II and ISO 27001 certification reports upon request under NDA. These reports should cover specifically the business version of their product (not just consumer services) and include all data centers where your information might be stored. Check the certification date to ensure it's within the last year.

Data Transfer

Data Storage Locations Identified

What this means: The vendor must clearly disclose where your data is physically stored.
What to look for:

• Documentation of all data center locations
• Confirmation that data is stored in jurisdictions with adequate data protection
• Information about which data types are stored in which locations
• Whether data can be restricted to specific regions if needed

Example: Google Workspace allows customers to choose data regions for certain services. In their documentation, they clearly list which services support regional storage (like Gmail and Drive) and which don't. They provide a map of all data center locations and specify which data types are stored in which locations.

Compliant Transfer Mechanisms

What this means: If data is transferred internationally, appropriate legal safeguards must be in place.
What to look for:

• Use of Standard Contractual Clauses (SCCs)
• Adherence to EU-US Data Privacy Framework
• Binding Corporate Rules if applicable
• Documentation of the specific transfer mechanisms used

Example: Adobe Creative Cloud incorporates the EU Standard Contractual Clauses (SCCs) into their data processing agreements. Their privacy documentation explicitly states which transfer mechanism applies to which data flows, such as using SCCs for EU to US transfers and identifying which services participate in the EU-US Data Privacy Framework.

Third-Country Transfer Controls

What this means: Additional protections for data transferred to countries without adequate data protection laws.
What to look for:

• Supplementary measures beyond standard transfer mechanisms
• Risk assessments for transfers to high-risk jurisdictions
• Ability to restrict transfers to specific countries if necessary
• Documentation of how data is protected in third countries

Example: ServiceNow provides a "Data Isolation Service" that allows customers to specify that their data must not be transferred to specific countries. Their documentation should detail additional encryption, access controls, and technical safeguards applied to data that must be transferred to countries without adequate protection laws.

Security Management

SSO Compatibility

What this means: The application should support Single Sign-On to integrate with your company's identity management.
What to look for:

• Support for industry-standard SSO protocols (SAML, OAuth, OpenID Connect)
• Compatibility with your specific identity provider
• Documentation on how to configure SSO
• No security compromises when SSO is enabled

Example: Atlassian's Jira Cloud supports SAML 2.0 for SSO integration with identity providers like Okta, Azure AD, and Google Workspace. Their documentation includes step-by-step guides for each major provider, explaining certificate requirements and attribute mapping. Check that enabling SSO doesn't bypass other security controls.

Multi-Factor Authentication

What this means: The application should require multiple forms of verification before granting access.
What to look for:

• Support for various authentication factors (something you know, have, and are)
• Options for SMS, email, authenticator apps, or hardware tokens
• Ability to enforce MFA for all users or specific roles
• Proper implementation that doesn't create security vulnerabilities

Example: Zoom allows administrators to enforce MFA for all users or specific user groups. They support multiple second factors including TOTP authenticator apps, SMS, and push notifications. Their admin console provides reports on MFA adoption and compliance, and allows recovery options to be configured to prevent lockouts.

Access Control Granularity

What this means: The application should allow detailed control over who can access what resources.
What to look for:

• Role-based access control with multiple predefined roles
• Ability to create custom roles with specific permissions
• Controls at various levels (system, feature, record, field)
• Audit logs of permission changes and access attempts

Example: Zendesk allows for hierarchical access control with predefined roles like agent, admin, and end-user, but also supports custom roles. Permissions can be set at different levels including ticket visibility, field-level access, and feature access. The audit log shows who changed permissions and when, enabling regular access review.

Section II: Operational and Technical Risk Management (High Criticality)

Internal Systems

Admin Privilege Levels Defined

What this means: The application should have clear tiers of administrative access.
What to look for:

• Multiple admin roles with different capabilities
• Principle of least privilege implementation
• Documentation of each role's permissions
• Ability to customize admin roles if needed

Example: Microsoft Azure provides multiple built-in administrative roles such as Global Administrator, Security Administrator, and Billing Administrator. Each role has clearly documented permissions, and the system supports custom roles that can be defined with specific permissions. Check that Azure Privileged Identity Management can be used to provide just-in-time admin access.

Access Control Policies

What this means: The application should enforce rules about who can access what and under what circumstances.
What to look for:

• Ability to set policies based on user attributes, time, location, etc.
• Features to enforce separation of duties
• Automatic enforcement of policies
• Regular policy review capabilities

Example: Okta allows for policy-based access control, including conditions based on network location, device posture, time of day, and user risk score. Their system can enforce separation of duties through group membership exclusions and supports automated policy enforcement. Policy reports show effectiveness and potential policy conflicts.

Logging and Monitoring Capabilities

What this means: The application should record important security events and allow for their review.
What to look for:

• Comprehensive logging of access, changes, and security events
• Log integrity protections
• Integration with SIEM or monitoring tools
• Alerting capabilities for suspicious activities

Example: Splunk Cloud captures comprehensive logs including authentication attempts, configuration changes, and data access events. The logs are stored with integrity controls to prevent tampering, support integration with popular SIEM tools, and include pre-built alerting for unusual patterns like after-hours admin access or geographic anomalies.

Backup and Recovery

Backup Frequency and Retention

What this means: The application should protect data through regular backups kept for appropriate periods.
What to look for:

• Automated backup schedule (daily, hourly, etc.)
• Configurable retention periods
• Different backup types (full, incremental)
• Documentation of what exactly is backed up

Example: Salesforce provides automated daily backups with a 30-day retention period by default. Their "Data Recovery" service allows for custom retention policies of up to 10 years for regulatory compliance. They should clearly document which elements are included in backups (data, metadata, attachments) and any excluded items.

Disaster Recovery Procedures

What this means: The application should have formal processes to restore functionality after a disaster.
What to look for:

• Documented disaster recovery plan
• Regular testing of recovery procedures
• Clear roles and responsibilities during recovery
• Multiple recovery scenarios addressed

Example: Oracle Cloud Infrastructure maintains a documented disaster recovery plan that covers scenarios from single component failure to complete data center loss. They conduct quarterly recovery exercises, publish an annual DR testing report, and maintain 24/7 incident response teams. Their DR documentation should specify team roles and communication procedures during incidents.

Recovery Time Objective (RTO)

What this means: The maximum acceptable time to restore the application after an outage.
What to look for:

• Clearly defined RTO in service agreements
• Evidence that the vendor can meet the stated RTO
• Different RTOs for different severity levels
• History of meeting RTOs during actual incidents

Example: IBM Cloud Databases defines tiered RTOs in their SLA: 15 minutes for critical services, 1 hour for high-priority services, and 4 hours for standard services. Their service status page shows historical recovery times during actual incidents, and they should provide evidence of regular testing to verify RTO commitments can be met.

Technical Implementation

Integration Test Results

What this means: Evidence that the application has been tested for compatibility with other systems.
What to look for:

• Test reports showing integration with similar systems
• Testing in environments similar to yours
• Documented test methodologies
• Results of performance tests during integration

Example: Workday provides detailed integration validation reports showing successful testing with popular HRIS systems, payroll providers, and SSO solutions. Their documentation includes performance metrics during API-heavy operations and outlines their test methodology, including automated regression testing for each update.

Technical Compatibility

What this means: The application should work with your existing technology stack.
What to look for:

• Compatibility with your operating systems, browsers, databases
• Support for required API standards
• Hardware requirements that match your infrastructure
• No conflicts with other critical applications

Example: Tableau Desktop publishes a detailed compatibility matrix showing supported operating systems (Windows 10/11, macOS versions), minimum hardware requirements, compatible browsers, and supported database connectors. They document which REST API versions are supported and any known conflicts with antivirus software or VPN solutions.

API Security Standards

What this means: Application programming interfaces should implement security best practices.
What to look for:

• Authentication requirements for API access
• Rate limiting to prevent abuse
• Input validation and output encoding
• API documentation addressing security

Example: Stripe's API requires authentication with API keys or OAuth tokens, implements rate limiting that gradually increases with account history, validates all inputs with clear error messages, and provides comprehensive security documentation including best practices for handling API keys and webhook signatures.

Extended Regulatory Compliance

Industry-Specific Standards

What this means: The application should comply with regulations specific to your industry.
What to look for:

• HIPAA compliance for healthcare
• PCI DSS for payment processing
• SOX for financial reporting
• Other industry-specific regulations relevant to your business

Example: Epic's electronic health record system provides a HIPAA compliance package including their Business Associate Agreement, documentation of technical safeguards, audit controls, and encryption methods. They maintain HITRUST certification, publish annual security assessments, and have a dedicated compliance team for healthcare regulations.

Compliance Certificates

What this means: Formal documentation proving compliance with required standards.
What to look for:

• Certificates issued by recognized authorities
• Coverage of relevant standards
• Current validity (not expired)
• Scope that includes the specific application being evaluated

Example: SAP Concur maintains PCI DSS Level 1 compliance certification (the highest level) for its expense management solution. The certificate should be current (renewed annually), issued by a Qualified Security Assessor (QSA), and specifically cover the Concur platform rather than just SAP generally.

Regulatory Documentation

What this means: The application should have documentation detailing how it meets regulatory requirements.
What to look for:

• Compliance statements for relevant regulations
• Documented controls mapped to regulatory requirements
• Regular compliance assessments
• Updates based on regulatory changes

Example: DocuSign maintains a regulatory compliance guide that maps their controls to specific requirements in regulations like GDPR, HIPAA, and SOX. They provide documentation showing how electronic signatures meet legal requirements in different jurisdictions and publish regular updates when regulations change.

Section III: Vendor and Performance Evaluation (Medium Criticality)

Licensing & Financial

Licensing Terms Reviewed

What this means: The legal terms governing the use of the software should be evaluated.
What to look for:

• Clear definition of permitted uses
• Limitations on users, transactions, or data volume
• Ownership of data created in the application
• Termination conditions and data retrieval options

Example: Adobe Creative Cloud Enterprise licensing agreement defines permitted use cases (commercial vs. educational), specifies the number of allowed installations per license, explicitly states that customers retain ownership of their created content, and outlines a 30-day period post-termination for data retrieval.

Subscription Models Assessed

What this means: The pricing structure should be evaluated for alignment with your needs.
What to look for:

• Pricing based on relevant metrics (users, data, transactions)
• Scalability of pricing as usage grows
• Hidden costs or fees
• Comparison with industry standard pricing

Example: Zoom offers tiered pricing based on number of hosts and meeting participants. Check for additional costs such as webinar capabilities, cloud recording storage, and phone services that may not be included in the base subscription. Compare with competitors like Microsoft Teams and Cisco Webex for equivalent feature sets.

Total Cost of Ownership Analysis

What this means: Evaluation of all costs associated with the application over its lifecycle.
What to look for:

• Direct costs (licenses, support)
• Indirect costs (implementation, training, maintenance)
• Opportunity costs of alternatives
• Expected ROI calculations

Example: Salesforce implementation requires consideration of license costs, implementation consulting (typically 1-2x license cost), annual administrator training, potential third-party add-ons, data migration costs, and ongoing customization expenses. A complete TCO should project these costs over 3-5 years and compare with alternatives.

Vendor Stability

Financial Statements Reviewed

What this means: Evaluation of the vendor's financial health to ensure long-term viability.
What to look for:

• Positive revenue growth
• Profitability or clear path to profitability
• Sustainable debt levels
• Sufficient cash reserves

Example: When evaluating a SaaS vendor like Monday.com, review their most recent 10-K or annual report to check revenue growth rate (e.g., >20% year-over-year), gross margin percentage (>70% is good for SaaS), debt-to-equity ratio (<0.5 is often favorable), and cash reserves (sufficient to cover at least 12 months of operations).

Market Reputation Assessed

What this means: Research into how the vendor is perceived in the marketplace.
What to look for:

• Customer reviews and testimonials
• Industry analyst reports (Gartner, Forrester)
• Media coverage and industry awards
• History of security incidents or breaches

Example: Evaluate Slack's market position by checking their placement in the Gartner Magic Quadrant for Workplace Collaboration, reading G2 and Capterra reviews (look for patterns, not just ratings), researching their response to their 2015 data breach, and reviewing case studies from companies similar to yours.

Business Continuity Plans Verified

What this means: Confirmation that the vendor can maintain service during disruptions.
What to look for:

• Documented business continuity plan
• Redundant infrastructure
• Staff succession planning
• Regular testing of continuity procedures

Example: Microsoft Azure provides documentation on their business continuity and disaster recovery capabilities, including globally distributed data centers, automatic failover procedures, and redundant network paths. They should provide evidence of regular DR exercises (at least quarterly) and publish a BC/DR test report annually.

Deployment Strategy

Central Management Features

What this means: Ability to administer the application from a centralized location.
What to look for:

• Central console for user management
• Deployment controls for updates
• Configuration management tools
• Centralized monitoring and alerts

Example: Microsoft Intune provides a central console for managing all organizational devices, allows for controlled rollout of application updates by user group, supports configuration profiles that can be applied to device categories, and offers a unified alert system for security and compliance issues.

MDM Compatibility

What this means: Integration with Mobile Device Management solutions if applicable.
What to look for:

• Support for major MDM platforms
• Security controls via MDM
• Application deployment through MDM
• Remote wipe capabilities

Example: Outlook for iOS and Android integrates with MDM platforms like Microsoft Intune, VMware Workspace ONE, and MobileIron. It supports security controls like requiring application PIN, preventing data copying between apps, and allowing selective remote wipe of corporate data without affecting personal data.

Automation Capabilities

What this means: Ability to automate routine tasks within the application.
What to look for:

• API access for automation
• Native workflow automation features
• Scheduling capabilities
• Integration with automation platforms

Example: ServiceNow provides extensive automation through its Flow Designer for visual workflow creation, comprehensive REST API for external integration, scheduled job capabilities for recurring tasks, and pre-built integrations with automation platforms like Power Automate and Zapier.

Section IV: User Experience and Performance (Lower Criticality)

Performance Metrics

Performance Benchmarks

What this means: Measured performance against industry standards or requirements.
What to look for:

• Response time metrics
• Throughput capabilities
• Resource utilization statistics
• Comparison with competitors or previous versions

Example: MongoDB Atlas should provide performance benchmarks showing response times for different query types, throughput metrics for various instance sizes, and comparisons with self-hosted MongoDB. Look for documentation showing performance under different loads and how it scales with additional resources.

Scalability Potential

What this means: Ability to handle increased load without degradation.
What to look for:

• Architectural support for scaling (horizontal/vertical)
• Performance at different user/data volumes
• Known scaling limitations
• Customer examples at your expected scale

Example: Shopify Plus documents its scalability capabilities including handling 10,000+ orders per minute, automatic scaling during traffic spikes, and supporting stores with millions of products. They should provide reference customers of similar size to your projected needs and clear documentation on any scaling limitations.

Load Testing Results

What this means: Evidence of performance under stress conditions.
What to look for:

• Test reports showing maximum capacity
• Performance at peak load
• Failure points and degradation patterns
• Recovery after excessive load

Example: Elastic Cloud provides load testing results showing performance with different index sizes and query loads. Look for documentation of testing methodologies, performance degradation patterns as capacity is reached, and automatic recovery capabilities after load spikes.

User Enablement

Training Resources Availability

What this means: Materials and support to help users learn the application.
What to look for:

• Documentation quality and comprehensiveness
• Video tutorials and guided tours
• Training programs (online or in-person)
• Knowledge base and community forums

Example: HubSpot offers a comprehensive training ecosystem including HubSpot Academy (free certification courses), detailed knowledge base articles, interactive guided tours within the product, weekly webinars, and an active community forum. Evaluate whether materials are updated for recent features and available in relevant languages.

Support SLA Quality

What this means: The vendor's commitments to resolving issues.
What to look for:

• Response time guarantees for different severity levels
• Support channels available (phone, email, chat)
• Hours of operation
• Escalation procedures

Example: Zendesk's Enterprise support plan includes a 1-hour response time for critical issues, 24/7 support availability, multiple channels (phone, email, chat), and a defined escalation path to senior support engineers and account managers. Compare these commitments to industry standards and your operational requirements.

Accessibility Compliance

What this means: The application should be usable by people with disabilities.
What to look for:

• Compliance with WCAG 2.1 standards
• Screen reader compatibility
• Keyboard navigation support
• Color contrast and text sizing options

Example: Microsoft Teams provides an accessibility conformance report detailing compliance with WCAG 2.1 AA standards, supports major screen readers (JAWS, NVDA, VoiceOver), allows complete keyboard navigation, and includes accessibility features like high contrast mode and adjustable text size. Check for regular accessibility testing and updates.

Disclaimer: This guide is provided for informational purposes only and does not constitute legal, security, or compliance advice. While we aim to offer practical considerations for evaluating third-party applications, your organization is solely responsible for conducting its own due diligence, risk assessments, and compliance reviews based on its internal policies and applicable laws or regulations.

We make no warranties or guarantees regarding the accuracy, completeness, or suitability of this content for your specific use case. Use of this guide does not create any legal obligation, client relationship, or guarantee of compliance outcomes.

For specific legal or compliance concerns, we recommend consulting with qualified legal, security, or compliance professionals.

• Evaluation tracker.xlsx10 KB