WordPress Bot & Crawler Detection

Detect crawlers and AI bots that never run the JavaScript SDK by dropping a small must-use plugin into your WordPress site. It forwards each front-end request's signals to Kitbase, which classifies the actor and stores bot/crawler traffic with attribution. Human requests are ignored.

A must-use plugin (mu-plugin) runs on every request, loads before regular plugins, and can't be deactivated from the admin — so your bot coverage survives theme switches and plugin changes. You can also paste the same snippet into your theme's functions.php, but an mu-plugin is theme-independent and recommended.

Privacy — we only keep the bots

Forwarding every request doesn't mean every request is stored. Human visitors' signals are used only to classify the request in memory and are then discarded — only bot and crawler requests are persisted. For those, the raw IP is stored only when IP logging is enabled for the project; otherwise it's used to derive geolocation (country, region, city) and then dropped.

Prerequisites

Add a constant to wp-config.php (above the /* That's all, stop editing! */ line):

define( 'KITBASE_API_KEY', 'sk_kitbase_…' ); // your project's SECRET key, not the browser SDK key

Setup

Create wp-content/mu-plugins/kitbase-crawler.php (make the mu-plugins folder if it doesn't exist). The request is sent non-blocking, so it never adds latency to your page load.

<?php
/**
 * Plugin Name: Kitbase Crawler Detection
 * Description: Forwards front-end request signals to Kitbase for server-side bot & crawler detection.
 */

if ( ! defined( 'ABSPATH' ) ) {
	exit;
}

add_action( 'template_redirect', function () {
	// Only forward real front-end page requests.
	if ( is_admin() || wp_doing_cron() || wp_doing_ajax() ) {
		return;
	}

	$server = $_SERVER;

	// Behind a proxy/CDN, the visitor IP is the first entry of X-Forwarded-For.
	$ip = $server['REMOTE_ADDR'] ?? null;
	if ( ! empty( $server['HTTP_X_FORWARDED_FOR'] ) ) {
		$ip = trim( explode( ',', $server['HTTP_X_FORWARDED_FOR'] )[0] );
	}

	$body = array(
		'events' => array(
			array(
				'user_agent'      => $server['HTTP_USER_AGENT'] ?? null,
				'ip_address'      => $ip,
				'method'          => $server['REQUEST_METHOD'] ?? null,
				'host'            => $server['HTTP_HOST'] ?? null,
				'path'            => wp_parse_url( $server['REQUEST_URI'] ?? '', PHP_URL_PATH ),
				'referrer'        => $server['HTTP_REFERER'] ?? null,
				'signature'       => $server['HTTP_SIGNATURE'] ?? null,
				'signature_input' => $server['HTTP_SIGNATURE_INPUT'] ?? null,
				'signature_agent' => $server['HTTP_SIGNATURE_AGENT'] ?? null,
			),
		),
	);

	// Fire-and-forget: blocking => false never delays the response.
	wp_remote_post( 'https://ingest.kitbase.dev/ingest/v1/server', array(
		'blocking'  => false,
		'timeout'   => 1,
		'headers'   => array(
			'Authorization' => 'Bearer ' . KITBASE_API_KEY,
			'Content-Type'  => 'application/json',
		),
		'body'      => wp_json_encode( $body ),
	) );
} );

That's it — no build step, no admin screen. The next crawler to hit your site shows up in Kitbase.

The signature* headers are part of Web Bot Auth — forwarding them lets Kitbase cryptographically verify a crawler's claimed identity when present.

High-traffic sites

wp_remote_post opens one non-blocking HTTP request per page view. On busy sites, prefer buffering signals (e.g. to an object cache or a log) and shipping them in batches of up to 500 on a short interval — see the nginx guide for the log-and-ship pattern.

Next steps

• API reference — full request schema, response, and attribution fields.
• All platforms — setup guides for other frameworks and hosts.