Skip to main content

API - Google Cloud

Nick Gusto
  • Updated

Team: Huntress Managed Security Information and Event Management (SIEM)
Product: Google Cloud
Environment: Cloud
Summary: Configure Google Cloud log ingestion for Huntress Managed SIEM using project scope or organization scope.

 

In this Article

Prerequisites and Important Notes
Choose a Scope
Setup Steps
Additional Notes
Troubleshooting

 

Field Value
Vendor  Google Cloud
Collection Method  API (pull)
Query Syntax event.provider = GCP
Source Types  Cloud Audit Logs (Admin Activity, Data Access, System Event, Policy Denied)
Billable Sources Calculation  Project scope: 1 source. Organization scope: 1 source for the organization, plus 1 additional source per GCP project discovered under it.
Additional Information  Can be configured at both the account and organization level (Huntress-side). Also supports both project-scoped and GCP-organization-scoped ingestion.

 

Prerequisites and Important Notes

Before configuring Google Cloud log ingestion, review the architecture requirements and required access permissions.

  • Huntress pulls your logs using a pull subscription in your Google Cloud project. No push endpoints, OIDC tokens, or credential files are created in your project.

  • You choose a scope (project or organization) when creating the source. This setting cannot be changed after creation. To switch scopes, create a new source and deactivate the existing one.

  • Project-scoped sources capture audit logs only for that specific project. Organization-scoped sources capture organization-level activity, such as IAM policy grants and changes to Organization Policy constraints.

  • The identity running the setup script requires specific roles:

    • Project scope: roles/pubsub.admin and roles/logging.configWriter on the target project (Owner or Editor satisfies both).

    • Organization scope: roles/logging.admin on the organization, plus roles/pubsub.admin on the project hosting the shared Pub/Sub resources.

 

Choose a Scope

Select the appropriate scope for your Google Cloud integration based on your logging requirements.

  • Project scope: Creates a topic, a Log Router sink, and a pull subscription in a single project to forward audit logs generated within that project.

  • Google Cloud organization scope: Creates one aggregated Log Router sink at the organization level using --include-children. Every current and future project forwards logs automatically without per-project setup. The Pub/Sub topic and subscription reside in a designated host project.

  • Organization scope is required to capture organization-level audit events, such as IAM policy bindings or Organization Policy changes made at the organization resource level.

 

Setup Steps

Follow these steps to create a Google Cloud source in Huntress and execute the setup script in Google Cloud Shell.

  1. Log in to Huntress and go to Source Management > Categories > Google Cloud.

  2. Select Create Source.

  3. Under Scope, select A single GCP project or A GCP organization (all projects, current and future).

  4. Enter a name and description for the source.

  5. Select Save.

  6. Select Open Google Cloud Shell to open Cloud Shell in a new tab.

  7. In Cloud Shell, confirm your working project by running gcloud config get-value project.

    • Project scope: Confirm this is the project you want to collect logs from. If incorrect, run gcloud config set project <your_project_id>.

    • Organization scope: Select any project designated to host the shared Pub/Sub resources.

  8. (Optional) In the Huntress tab, select View the setup script to review the script actions.

  9. In the Huntress tab, copy the setup command (curl -sSL '...' | bash), then paste and run it in Cloud Shell.

  10. Organization scope only: Enter your Google Cloud Organization ID (for example, <your_organization_id>) when prompted.

  11. Wait for the script to execute pre-flight checks, create the Pub/Sub topic, sink, and subscription, and grant roles/pubsub.subscriber to the Huntress service account.

  12. Confirm the validation summary displays green checkmarks and no errors.

  13. Return to the Huntress tab and verify that the source status updates from Pending to Configured, then to Active, when logs arrive.

 

Additional Notes

Review these operational details regarding log filtering, billing calculations, and resource management.

  • By default, the Log Router sink forwards Cloud Audit Logs (logName:"cloudaudit.googleapis.com"). Admin Activity logs are enabled by default. Data Access logs must be enabled under IAM & Admin > Audit Logs.

  • In the organization scope, the organization resource and each discovered project are billed as separate sources after Huntress receives at least one log from them.

  • Huntress creates only the Pub/Sub topic, sink, and subscription in your project and authenticates using a dedicated service account scoped strictly to read that subscription.

 

Troubleshooting

If Google Cloud log data does not appear in Huntress Managed SIEM, work through the following checks.

  • Source status is Pending or Configured: Pending indicates the setup script has not finished. Configured indicates the script completed, but no logs have arrived yet. Generate activity in the project to trigger an audit log entry.

  • Re-run the setup script: It is safe to re-run the script because resources are created only if they do not already exist.

  • Verify pull subscription: In Google Cloud Console, go to Pub/Sub > Subscriptions and confirm no push endpoint is configured and that the Huntress service account has roles/pubsub.subscriber.

  • Confirm sink filter matches: Verify the sink filter matches Cloud Audit Logs.

  • Check publish permissions: In Logging > Log Router > huntress-logs-sink, verify the writer identity has roles/pubsub.publisher on the huntress-logs topic.

  • Verify organization scope sink: Run gcloud logging sinks describe huntress-logs-sink --organization=<your_organization_id> and confirm includeChildren: true.

  • Missing organization-level events: Confirm that the source is configured for the Google Cloud organization scope rather than the project scope.

If you encounter an error that looks like this:

ERROR: (gcloud.pubsub.subscriptions.add-iam-policy-binding) FAILED_PRECONDITION: One or more users named in the policy do not belong to a permitted customer.
- '@type': type.googleapis.com/google.rpc.PreconditionFailure
  violations:
  - description: User huntress-logs-pull@siem-502418.iam.gserviceaccount.com is not
      in permitted organization.
    subject: orgpolicy:projects/test-project-1337/subscriptions/huntress-logs-sub?configvalue=huntress-logs-pull%40siem-502418.iam.gserviceaccount.com
    type: constraints/iam.allowedPolicyMemberDomains

 

You will need to modify your Domain Restricted Sharing policy. This can be done through the Google Cloud Console or Cloud Shell Editor.

 

Google Cloud Console

  • Go to IAM & Admin > Organization Policies in the Cloud Console.
  • Search for Domain Restricted Sharing (constraints/iam.allowedPolicyMemberDomains) and click it.
  • Click Edit Policy at the top of the page.
  • Select Override parent's policy.
  • Select "Replace": Under Policy enforcement, click the radio button for Replace. This stops the project from inheriting the parent organization's domain restrictions.
  • Add a Rule: Click the + Add a rule button.
  • Allow All: In the rule settings that appear, set the policy value to Allow all.
  • Save: Click Set policy at the bottom left.

Cloud Shell Editor

  • Create the new policy by pasting the following into your Cloud Shell:
cat <<EOF > allow_domains.yaml
constraint: constraints/iam.allowedPolicyMemberDomains
listPolicy:
  allValues: ALLOW
EOF
  • Once the policy has been created, run the following command:
gcloud resource-manager org-policies set-policy allow_domains.yaml --project=replace-with-your-project-name
  • Re-run the SIEM setup script.