Hybrid keyword data: Google Ads coverage for non-Labs countries + opt-in clickstream volumes (#262)
This commit is contained in:
Ben Senescu
GitHub
parent
e41b52ca56
commit
b809f87b5e
110
specs/0004-keyword-data-source-routing.md
Normal file
110
specs/0004-keyword-data-source-routing.md
Normal file
@ -0,0 +1,110 @@
|
||||
# Keyword data source routing and the clickstream default
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (June 2026)
|
||||
|
||||
## Context
|
||||
|
||||
Keyword research is the largest credit spend for OpenSEO users, and coverage
|
||||
has a hard gap: DataForSEO Labs supports 94 countries, so a customer in
|
||||
Iceland (location 2352) cannot run keyword research at all.
|
||||
|
||||
Three data sources were on the table:
|
||||
|
||||
- **DataForSEO Labs** — our existing source. Per-row pricing ($0.01/task +
|
||||
$0.0001/row), and the only source for keyword difficulty, search intent,
|
||||
and SERP-feature context. Its `include_clickstream_data` flag doubles the
|
||||
request cost; its only effect is refined volume numbers — the standard
|
||||
`keyword_info.search_volume` is the same Google-Ads-derived volume every
|
||||
mainstream tool shows.
|
||||
- **DataForSEO Keywords Data (Google Ads endpoints)** — same vendor, flat
|
||||
$0.075 per live request (up to 1,000 keywords for `search_volume`, up to
|
||||
20 seeds for `keywords_for_keywords`), 217 countries including Iceland. No
|
||||
difficulty, intent, or SERP data; volumes are bucketed and aggregate close
|
||||
variants.
|
||||
- **Direct Google Ads API** — free, but not usable in a SaaS: Google's
|
||||
Targeting-data policy forbids collecting Keyword Planner data "for any
|
||||
purposes other than creating or managing Google Ads campaigns," and
|
||||
exposing it to users requires the full Required Minimum Functionality (a
|
||||
campaign-management suite). Volumes are bucketed without active ad spend.
|
||||
|
||||
## Decision
|
||||
|
||||
Every supported country has exactly one keyword-data provider, resolved by
|
||||
`getKeywordDataProvider(locationCode)` in `src/shared/keyword-locations.ts`.
|
||||
There is no user-facing provider choice.
|
||||
|
||||
1. **Labs is the default provider.** Countries Labs does not cover are
|
||||
flagged `googleAdsOnly` in `LOCATION_OPTIONS` and are served by the
|
||||
Keywords Data Google Ads endpoints. Unknown location codes fall back to
|
||||
Labs, which rejects them with its own error.
|
||||
2. **Routing per feature:**
|
||||
|
||||
| Feature | Labs country | Google-Ads-only country (e.g. Iceland) |
|
||||
| --------------------------------------------------- | ------------------------------ | ----------------------------------------------------------------------------------------------- |
|
||||
| Keyword research (UI + `research_keywords`) | Labs related→suggestions→ideas | `keywords_for_keywords` (single source) |
|
||||
| `get_keyword_metrics`, rank-tracking metric refresh | Labs keyword_overview | `search_volume` |
|
||||
| SERP analysis, `get_serp_results`, rank tracking | SERP API | SERP API (supports all countries) |
|
||||
| Domain overview, ranked keywords, SERP competitors | Labs | **Unavailable** — pickers filtered to Labs countries; MCP tools return a clear validation error |
|
||||
|
||||
3. **Google-Ads-sourced rows carry no keyword difficulty or intent**
|
||||
(`keywordDifficulty: null`, `intent: "unknown"`). The research page and
|
||||
the MCP tool descriptions state this whenever such a country is in play.
|
||||
4. **Clickstream refinement is opt-in per call and defaults to off**, for
|
||||
the Labs research endpoints (related/suggestions/ideas) and keyword
|
||||
overview. Users opt in via the labeled checkbox on the research page
|
||||
(URL param `cs`, carried per keyword tab, hidden for Google-Ads-only
|
||||
countries) or via `includeClickstreamData` on the `research_keywords` and
|
||||
`get_keyword_metrics` MCP tools. The label and tool descriptions must
|
||||
state the 2× credit cost. The flag is part of the research cache key.
|
||||
5. **Language codes for Google-Ads-only countries must exist in both the
|
||||
Google Ads and SERP language lists** — the country picker is shared with
|
||||
rank tracking, which uses the SERP API. China is excluded: its Ads
|
||||
language code (`zh_CN`) conflicts with the SERP format (`zh-CN`), and
|
||||
Google search does not meaningfully operate there.
|
||||
6. **Billing is unchanged.** `keywords_data/*` task costs flow through the
|
||||
same envelope → markup → Autumn pipeline as Labs calls and map to the
|
||||
`keyword_research` credit feature (rank tracking overrides to
|
||||
`rank_tracking`).
|
||||
|
||||
## Rationale
|
||||
|
||||
Cost at our actual defaults (research default = 150 rows/seed; credits =
|
||||
USD × 1.28 markup × 1000):
|
||||
|
||||
| Call | Labs (with clickstream) | Labs (default) | Google Ads |
|
||||
| ------------------ | ----------------------- | -------------- | -------------- |
|
||||
| research, 150 rows | $0.050 → 64 cr | $0.025 → 32 cr | $0.075 → 96 cr |
|
||||
| research, 500 rows | $0.120 → 154 cr | $0.060 → 77 cr | $0.075 → 96 cr |
|
||||
| metrics, 100 kw | $0.020 → 26 cr | same | $0.075 → 96 cr |
|
||||
| metrics, 700 kw | $0.080 → 103 cr | same | $0.075 → 96 cr |
|
||||
|
||||
- A wholesale switch to Google Ads data would raise the cost of typical
|
||||
calls and lose difficulty/intent; replicating difficulty alone via
|
||||
`bulk_keyword_difficulty` ($0.11/1k) erases any savings. Hybrid keeps the
|
||||
better data where it exists and adds coverage where it doesn't.
|
||||
- Always-on clickstream silently doubled the #1 spend feature for a marginal
|
||||
volume refinement. Off-by-default halves default research cost
|
||||
(~64 → ~32 credits per seed); the opt-in keeps the refinement available to
|
||||
users who want it, priced visibly.
|
||||
- The direct Google Ads API is rejected on policy, not effort — revisit only
|
||||
if OpenSEO ships campaign management.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Iceland and ~47 other countries are selectable for keyword research and
|
||||
rank tracking; a Google-Ads-served research or metrics call costs a flat
|
||||
~96 credits.
|
||||
- Cross-country volume numbers stay roughly comparable: both providers'
|
||||
standard volumes derive from Google Ads data.
|
||||
- Google Ads live endpoints allow 12 requests/min per DataForSEO account.
|
||||
Research fans out at most 5 seeds per call, so a single user stays under
|
||||
it; sustained multi-user traffic on these countries would queue.
|
||||
- `keywords_for_keywords` has no limit parameter (up to 20k suggestions per
|
||||
flat-fee request); results are sorted by volume server-side and truncated
|
||||
to the requested limit.
|
||||
- The research cache version was bumped (2→3) so pre-change
|
||||
clickstream-priced volumes never mix with standard ones.
|
||||
- Reverting the clickstream default is a one-line change per fetcher in
|
||||
`src/server/lib/dataforseo/labs.ts`; the opt-in plumbing stays either way.
|
||||
@ -11,7 +11,7 @@ import {
|
||||
DEFAULT_LOCATION_CODE,
|
||||
LOCATIONS,
|
||||
getLanguageCode,
|
||||
isSupportedLocationCode,
|
||||
isLabsLocationCode,
|
||||
} from "@/client/features/keywords/locations";
|
||||
import { useDomainSearchHistory } from "@/client/hooks/useDomainSearchHistory";
|
||||
import type { DomainSearchHistoryItem } from "@/client/hooks/useDomainSearchHistory";
|
||||
@ -124,7 +124,7 @@ function getHistorySearchUpdate(
|
||||
item: DomainSearchHistoryItem,
|
||||
): DomainSearchUpdate {
|
||||
const historyLocation =
|
||||
item.locationCode != null && isSupportedLocationCode(item.locationCode)
|
||||
item.locationCode != null && isLabsLocationCode(item.locationCode)
|
||||
? item.locationCode
|
||||
: DEFAULT_LOCATION_CODE;
|
||||
|
||||
|
||||
@ -4,7 +4,7 @@ import { getFieldError, getFormError } from "@/client/lib/forms";
|
||||
import type { DomainOverviewControlsForm } from "@/client/features/domain/DomainOverviewPage";
|
||||
import { toSortMode } from "@/client/features/domain/utils";
|
||||
import type { DomainSortMode } from "@/client/features/domain/types";
|
||||
import { LOCATION_OPTIONS } from "@/client/features/keywords/locations";
|
||||
import { LABS_LOCATION_OPTIONS } from "@/client/features/keywords/locations";
|
||||
|
||||
type Props = {
|
||||
controlsForm: DomainOverviewControlsForm;
|
||||
@ -63,7 +63,7 @@ export function DomainSearchCard({
|
||||
onLocationChange(next);
|
||||
}}
|
||||
>
|
||||
{LOCATION_OPTIONS.map((option) => (
|
||||
{LABS_LOCATION_OPTIONS.map((option) => (
|
||||
<option key={option.code} value={option.code}>
|
||||
{option.label}
|
||||
</option>
|
||||
|
||||
@ -4,7 +4,7 @@ import {
|
||||
} from "@/types/schemas/domain";
|
||||
import {
|
||||
DEFAULT_LOCATION_CODE,
|
||||
isSupportedLocationCode,
|
||||
isLabsLocationCode,
|
||||
} from "@/client/features/keywords/locations";
|
||||
import {
|
||||
EMPTY_DOMAIN_FILTERS,
|
||||
@ -46,8 +46,9 @@ export function getDomainRouteState(
|
||||
search: DomainSearchParams,
|
||||
): DomainOverviewRouteState {
|
||||
const normalizedSort = toSortMode(search.sort ?? null) ?? "traffic";
|
||||
// Domain analytics is Labs-backed; Google-Ads-only countries aren't valid.
|
||||
const normalizedLocationCode =
|
||||
search.loc != null && isSupportedLocationCode(search.loc)
|
||||
search.loc != null && isLabsLocationCode(search.loc)
|
||||
? search.loc
|
||||
: DEFAULT_LOCATION_CODE;
|
||||
|
||||
|
||||
@ -16,6 +16,7 @@ type KeywordTabValidationInput = {
|
||||
locationCode: number;
|
||||
resultLimit: ResultLimit;
|
||||
mode: KeywordMode;
|
||||
clickstream: boolean;
|
||||
};
|
||||
|
||||
type UseKeywordControlsFormInput = {
|
||||
@ -23,6 +24,7 @@ type UseKeywordControlsFormInput = {
|
||||
locationCode: number;
|
||||
resultLimit: ResultLimit;
|
||||
keywordMode: KeywordMode;
|
||||
clickstream: boolean;
|
||||
getOpenKeywordTabs?: () => readonly KeywordTabValidationInput[];
|
||||
keywordTabsLimit?: number;
|
||||
};
|
||||
@ -32,6 +34,7 @@ export type KeywordControlsValues = {
|
||||
locationCode: number;
|
||||
resultLimit: ResultLimit;
|
||||
mode: KeywordMode;
|
||||
clickstream: boolean;
|
||||
};
|
||||
|
||||
function getKeywordSearchValidationErrors(
|
||||
@ -82,6 +85,7 @@ function getKeywordTabCapacityError(
|
||||
locationCode: value.locationCode,
|
||||
resultLimit: value.resultLimit,
|
||||
mode: value.mode,
|
||||
clickstream: value.clickstream,
|
||||
};
|
||||
const alreadyOpen = simulatedOpenTabs.some((tab) =>
|
||||
keywordTabMatches(tab, input),
|
||||
@ -113,7 +117,8 @@ function keywordTabMatches(
|
||||
tab.keyword === input.keyword &&
|
||||
tab.locationCode === input.locationCode &&
|
||||
tab.resultLimit === input.resultLimit &&
|
||||
tab.mode === input.mode
|
||||
tab.mode === input.mode &&
|
||||
tab.clickstream === input.clickstream
|
||||
);
|
||||
}
|
||||
|
||||
@ -127,6 +132,7 @@ export function useKeywordControlsForm(
|
||||
locationCode: input.locationCode,
|
||||
resultLimit: input.resultLimit,
|
||||
mode: input.keywordMode,
|
||||
clickstream: input.clickstream,
|
||||
},
|
||||
validators: {
|
||||
onChange: ({ formApi, value }) =>
|
||||
@ -154,6 +160,7 @@ export function useKeywordControlsForm(
|
||||
locationCode: input.locationCode,
|
||||
resultLimit: input.resultLimit,
|
||||
mode: input.keywordMode,
|
||||
clickstream: input.clickstream,
|
||||
});
|
||||
}, [
|
||||
form,
|
||||
@ -161,6 +168,7 @@ export function useKeywordControlsForm(
|
||||
input.keywordMode,
|
||||
input.locationCode,
|
||||
input.resultLimit,
|
||||
input.clickstream,
|
||||
]);
|
||||
|
||||
return form;
|
||||
|
||||
@ -8,7 +8,7 @@ import { parseKeywordInput } from "@/client/features/keywords/state/keywordContr
|
||||
import { researchKeywords } from "@/serverFunctions/keywords";
|
||||
import type {
|
||||
KeywordMode,
|
||||
KeywordSource,
|
||||
ResearchSource,
|
||||
ResultLimit,
|
||||
} from "@/client/features/keywords/keywordResearchTypes";
|
||||
|
||||
@ -24,6 +24,7 @@ type KeywordResearchQueryInput = {
|
||||
locationCode: number;
|
||||
resultLimit: ResultLimit;
|
||||
mode: KeywordMode;
|
||||
clickstream: boolean;
|
||||
};
|
||||
|
||||
type KeywordResearchRequest = {
|
||||
@ -34,6 +35,7 @@ type KeywordResearchRequest = {
|
||||
languageCode: string;
|
||||
resultLimit: ResultLimit;
|
||||
mode: KeywordMode;
|
||||
clickstream: boolean;
|
||||
};
|
||||
|
||||
export const KEYWORD_RESEARCH_STALE_TIME_MS = 24 * 60 * 60 * 1000;
|
||||
@ -53,6 +55,7 @@ export function buildKeywordResearchRequest(
|
||||
languageCode: getLanguageCode(input.locationCode),
|
||||
resultLimit: input.resultLimit,
|
||||
mode: input.mode,
|
||||
clickstream: input.clickstream,
|
||||
};
|
||||
}
|
||||
|
||||
@ -68,6 +71,7 @@ export function buildKeywordResearchQueryKey(
|
||||
request.languageCode,
|
||||
request.resultLimit,
|
||||
request.mode,
|
||||
request.clickstream,
|
||||
]
|
||||
: ["keywordResearch", "idle"];
|
||||
}
|
||||
@ -81,6 +85,7 @@ export function keywordResearchQueryFn(request: KeywordResearchRequest) {
|
||||
languageCode: request.languageCode,
|
||||
resultLimit: request.resultLimit,
|
||||
mode: request.mode,
|
||||
clickstream: request.clickstream,
|
||||
},
|
||||
});
|
||||
}
|
||||
@ -89,7 +94,14 @@ export function useKeywordResearchData(
|
||||
input: KeywordResearchQueryInput,
|
||||
addSearch: AddSearchFn,
|
||||
) {
|
||||
const { keywordInput, locationCode, mode, projectId, resultLimit } = input;
|
||||
const {
|
||||
clickstream,
|
||||
keywordInput,
|
||||
locationCode,
|
||||
mode,
|
||||
projectId,
|
||||
resultLimit,
|
||||
} = input;
|
||||
const request = useMemo<KeywordResearchRequest | null>(
|
||||
() =>
|
||||
buildKeywordResearchRequest({
|
||||
@ -98,8 +110,9 @@ export function useKeywordResearchData(
|
||||
mode,
|
||||
projectId,
|
||||
resultLimit,
|
||||
clickstream,
|
||||
}),
|
||||
[keywordInput, locationCode, mode, projectId, resultLimit],
|
||||
[clickstream, keywordInput, locationCode, mode, projectId, resultLimit],
|
||||
);
|
||||
const queryKey = useMemo(
|
||||
() => buildKeywordResearchQueryKey(request),
|
||||
@ -133,6 +146,7 @@ export function useKeywordResearchData(
|
||||
captureClientEvent("keyword_research:search_complete", {
|
||||
location_code: request.locationCode,
|
||||
search_mode: request.mode,
|
||||
clickstream: request.clickstream,
|
||||
result_count: researchQuery.data.rows.length,
|
||||
});
|
||||
|
||||
@ -161,7 +175,7 @@ export function useKeywordResearchData(
|
||||
hasSearched,
|
||||
lastSearchError: hasSearched && researchQuery.isError,
|
||||
lastResultSource:
|
||||
researchQuery.data?.source ?? ("related" as KeywordSource),
|
||||
researchQuery.data?.source ?? ("related" as ResearchSource),
|
||||
lastUsedFallback: researchQuery.data?.usedFallback ?? false,
|
||||
lastSearchKeyword: request?.seedKeyword ?? "",
|
||||
lastSearchLocationCode: request?.locationCode ?? DEFAULT_LOCATION_CODE,
|
||||
|
||||
@ -5,6 +5,8 @@ export const RESULT_LIMITS: ResultLimit[] = [150, 300, 500];
|
||||
|
||||
export type KeywordSource = "related" | "suggestions" | "ideas";
|
||||
export type KeywordMode = "auto" | KeywordSource;
|
||||
/** Actual result source; google_ads serves countries Labs doesn't cover. */
|
||||
export type ResearchSource = KeywordSource | "google_ads";
|
||||
|
||||
export type KeywordFilterValues = {
|
||||
include: string;
|
||||
|
||||
@ -9,6 +9,7 @@ type KeywordSearchParams = {
|
||||
loc?: number;
|
||||
kLimit?: ResultLimit;
|
||||
mode?: KeywordMode;
|
||||
cs?: boolean;
|
||||
sort?: SortField;
|
||||
order?: SortDir;
|
||||
minVol?: string;
|
||||
@ -31,6 +32,7 @@ export function normalizeLegacyKeywordSearch(search: KeywordSearchParams): {
|
||||
loc: search.loc,
|
||||
kLimit: search.kLimit === 150 ? undefined : search.kLimit,
|
||||
mode: search.mode === "auto" ? undefined : search.mode,
|
||||
cs: search.cs === true ? true : undefined,
|
||||
sort: search.sort === "searchVolume" ? undefined : search.sort,
|
||||
order: search.order === "desc" ? undefined : search.order,
|
||||
minVol: undefined,
|
||||
@ -48,6 +50,7 @@ export function normalizeLegacyKeywordSearch(search: KeywordSearchParams): {
|
||||
"loc",
|
||||
"kLimit",
|
||||
"mode",
|
||||
"cs",
|
||||
"sort",
|
||||
"order",
|
||||
"minVol",
|
||||
|
||||
@ -1,156 +1,11 @@
|
||||
/**
|
||||
* DataForSEO-supported countries.
|
||||
*
|
||||
* Source: https://cdn.dataforseo.com/v3/locations/locations_and_languages_dataforseo_labs_2026_04_06.csv
|
||||
*
|
||||
* For countries with multiple Google-supported languages, we pick the
|
||||
* language with the largest keyword corpus (the primary search market)
|
||||
* as the default. DataForSEO Labs APIs accept a single location_code +
|
||||
* language_code pair per request, so we expose one entry per country.
|
||||
*
|
||||
* Entries are sorted alphabetically by country name; pick US as the
|
||||
* product-wide default via DEFAULT_LOCATION_CODE below.
|
||||
*/
|
||||
export const DEFAULT_LOCATION_CODE = 2840;
|
||||
|
||||
export const LOCATION_OPTIONS = [
|
||||
{ code: 2008, label: "Albania", shortLabel: "AL", languageCode: "sq" },
|
||||
{ code: 2012, label: "Algeria", shortLabel: "DZ", languageCode: "fr" },
|
||||
{ code: 2024, label: "Angola", shortLabel: "AO", languageCode: "pt" },
|
||||
{ code: 2032, label: "Argentina", shortLabel: "AR", languageCode: "es" },
|
||||
{ code: 2051, label: "Armenia", shortLabel: "AM", languageCode: "hy" },
|
||||
{ code: 2036, label: "Australia", shortLabel: "AU", languageCode: "en" },
|
||||
{ code: 2040, label: "Austria", shortLabel: "AT", languageCode: "de" },
|
||||
{ code: 2031, label: "Azerbaijan", shortLabel: "AZ", languageCode: "az" },
|
||||
{ code: 2048, label: "Bahrain", shortLabel: "BH", languageCode: "ar" },
|
||||
{ code: 2050, label: "Bangladesh", shortLabel: "BD", languageCode: "bn" },
|
||||
{ code: 2056, label: "Belgium", shortLabel: "BE", languageCode: "nl" },
|
||||
{ code: 2068, label: "Bolivia", shortLabel: "BO", languageCode: "es" },
|
||||
{
|
||||
code: 2070,
|
||||
label: "Bosnia and Herzegovina",
|
||||
shortLabel: "BA",
|
||||
languageCode: "bs",
|
||||
},
|
||||
{ code: 2076, label: "Brazil", shortLabel: "BR", languageCode: "pt" },
|
||||
{ code: 2100, label: "Bulgaria", shortLabel: "BG", languageCode: "bg" },
|
||||
{ code: 2854, label: "Burkina Faso", shortLabel: "BF", languageCode: "fr" },
|
||||
{ code: 2116, label: "Cambodia", shortLabel: "KH", languageCode: "en" },
|
||||
{ code: 2120, label: "Cameroon", shortLabel: "CM", languageCode: "fr" },
|
||||
{ code: 2124, label: "Canada", shortLabel: "CA", languageCode: "en" },
|
||||
{ code: 2152, label: "Chile", shortLabel: "CL", languageCode: "es" },
|
||||
{ code: 2170, label: "Colombia", shortLabel: "CO", languageCode: "es" },
|
||||
{ code: 2188, label: "Costa Rica", shortLabel: "CR", languageCode: "es" },
|
||||
{ code: 2384, label: "Cote d'Ivoire", shortLabel: "CI", languageCode: "fr" },
|
||||
{ code: 2191, label: "Croatia", shortLabel: "HR", languageCode: "hr" },
|
||||
{ code: 2196, label: "Cyprus", shortLabel: "CY", languageCode: "el" },
|
||||
{ code: 2203, label: "Czechia", shortLabel: "CZ", languageCode: "cs" },
|
||||
{ code: 2208, label: "Denmark", shortLabel: "DK", languageCode: "da" },
|
||||
{ code: 2218, label: "Ecuador", shortLabel: "EC", languageCode: "es" },
|
||||
{ code: 2818, label: "Egypt", shortLabel: "EG", languageCode: "ar" },
|
||||
{ code: 2222, label: "El Salvador", shortLabel: "SV", languageCode: "es" },
|
||||
{ code: 2233, label: "Estonia", shortLabel: "EE", languageCode: "et" },
|
||||
{ code: 2246, label: "Finland", shortLabel: "FI", languageCode: "fi" },
|
||||
{ code: 2250, label: "France", shortLabel: "FR", languageCode: "fr" },
|
||||
{ code: 2276, label: "Germany", shortLabel: "DE", languageCode: "de" },
|
||||
{ code: 2288, label: "Ghana", shortLabel: "GH", languageCode: "en" },
|
||||
{ code: 2300, label: "Greece", shortLabel: "GR", languageCode: "el" },
|
||||
{ code: 2320, label: "Guatemala", shortLabel: "GT", languageCode: "es" },
|
||||
{ code: 2344, label: "Hong Kong", shortLabel: "HK", languageCode: "zh-TW" },
|
||||
{ code: 2348, label: "Hungary", shortLabel: "HU", languageCode: "hu" },
|
||||
{ code: 2356, label: "India", shortLabel: "IN", languageCode: "en" },
|
||||
{ code: 2360, label: "Indonesia", shortLabel: "ID", languageCode: "id" },
|
||||
{ code: 2372, label: "Ireland", shortLabel: "IE", languageCode: "en" },
|
||||
{ code: 2376, label: "Israel", shortLabel: "IL", languageCode: "he" },
|
||||
{ code: 2380, label: "Italy", shortLabel: "IT", languageCode: "it" },
|
||||
{ code: 2392, label: "Japan", shortLabel: "JP", languageCode: "ja" },
|
||||
{ code: 2400, label: "Jordan", shortLabel: "JO", languageCode: "ar" },
|
||||
{ code: 2398, label: "Kazakhstan", shortLabel: "KZ", languageCode: "ru" },
|
||||
{ code: 2404, label: "Kenya", shortLabel: "KE", languageCode: "en" },
|
||||
{ code: 2428, label: "Latvia", shortLabel: "LV", languageCode: "lv" },
|
||||
{ code: 2440, label: "Lithuania", shortLabel: "LT", languageCode: "lt" },
|
||||
{ code: 2458, label: "Malaysia", shortLabel: "MY", languageCode: "en" },
|
||||
{ code: 2470, label: "Malta", shortLabel: "MT", languageCode: "en" },
|
||||
{ code: 2484, label: "Mexico", shortLabel: "MX", languageCode: "es" },
|
||||
{ code: 2498, label: "Moldova", shortLabel: "MD", languageCode: "ro" },
|
||||
{ code: 2492, label: "Monaco", shortLabel: "MC", languageCode: "fr" },
|
||||
{ code: 2504, label: "Morocco", shortLabel: "MA", languageCode: "ar" },
|
||||
{
|
||||
code: 2104,
|
||||
label: "Myanmar (Burma)",
|
||||
shortLabel: "MM",
|
||||
languageCode: "en",
|
||||
},
|
||||
{ code: 2528, label: "Netherlands", shortLabel: "NL", languageCode: "nl" },
|
||||
{ code: 2554, label: "New Zealand", shortLabel: "NZ", languageCode: "en" },
|
||||
{ code: 2558, label: "Nicaragua", shortLabel: "NI", languageCode: "es" },
|
||||
{ code: 2566, label: "Nigeria", shortLabel: "NG", languageCode: "en" },
|
||||
{
|
||||
code: 2807,
|
||||
label: "North Macedonia",
|
||||
shortLabel: "MK",
|
||||
languageCode: "mk",
|
||||
},
|
||||
{ code: 2578, label: "Norway", shortLabel: "NO", languageCode: "nb" },
|
||||
{ code: 2586, label: "Pakistan", shortLabel: "PK", languageCode: "en" },
|
||||
{ code: 2591, label: "Panama", shortLabel: "PA", languageCode: "es" },
|
||||
{ code: 2600, label: "Paraguay", shortLabel: "PY", languageCode: "es" },
|
||||
{ code: 2604, label: "Peru", shortLabel: "PE", languageCode: "es" },
|
||||
{ code: 2608, label: "Philippines", shortLabel: "PH", languageCode: "en" },
|
||||
{ code: 2616, label: "Poland", shortLabel: "PL", languageCode: "pl" },
|
||||
{ code: 2620, label: "Portugal", shortLabel: "PT", languageCode: "pt" },
|
||||
{ code: 2642, label: "Romania", shortLabel: "RO", languageCode: "ro" },
|
||||
{ code: 2682, label: "Saudi Arabia", shortLabel: "SA", languageCode: "ar" },
|
||||
{ code: 2686, label: "Senegal", shortLabel: "SN", languageCode: "fr" },
|
||||
{ code: 2688, label: "Serbia", shortLabel: "RS", languageCode: "sr" },
|
||||
{ code: 2702, label: "Singapore", shortLabel: "SG", languageCode: "en" },
|
||||
{ code: 2703, label: "Slovakia", shortLabel: "SK", languageCode: "sk" },
|
||||
{ code: 2705, label: "Slovenia", shortLabel: "SI", languageCode: "sl" },
|
||||
{ code: 2710, label: "South Africa", shortLabel: "ZA", languageCode: "en" },
|
||||
{ code: 2410, label: "South Korea", shortLabel: "KR", languageCode: "ko" },
|
||||
{ code: 2724, label: "Spain", shortLabel: "ES", languageCode: "es" },
|
||||
{ code: 2144, label: "Sri Lanka", shortLabel: "LK", languageCode: "en" },
|
||||
{ code: 2752, label: "Sweden", shortLabel: "SE", languageCode: "sv" },
|
||||
{ code: 2756, label: "Switzerland", shortLabel: "CH", languageCode: "de" },
|
||||
{ code: 2158, label: "Taiwan", shortLabel: "TW", languageCode: "zh-TW" },
|
||||
{ code: 2764, label: "Thailand", shortLabel: "TH", languageCode: "th" },
|
||||
{ code: 2788, label: "Tunisia", shortLabel: "TN", languageCode: "ar" },
|
||||
{ code: 2792, label: "Turkiye", shortLabel: "TR", languageCode: "tr" },
|
||||
{ code: 2804, label: "Ukraine", shortLabel: "UA", languageCode: "uk" },
|
||||
{
|
||||
code: 2784,
|
||||
label: "United Arab Emirates",
|
||||
shortLabel: "AE",
|
||||
languageCode: "en",
|
||||
},
|
||||
{
|
||||
code: 2826,
|
||||
label: "United Kingdom",
|
||||
shortLabel: "UK",
|
||||
languageCode: "en",
|
||||
},
|
||||
{ code: 2840, label: "United States", shortLabel: "US", languageCode: "en" },
|
||||
{ code: 2858, label: "Uruguay", shortLabel: "UY", languageCode: "es" },
|
||||
{ code: 2862, label: "Venezuela", shortLabel: "VE", languageCode: "es" },
|
||||
{ code: 2704, label: "Vietnam", shortLabel: "VN", languageCode: "vi" },
|
||||
] as const;
|
||||
|
||||
const LOCATION_CODES = new Set<number>(
|
||||
LOCATION_OPTIONS.map((option) => option.code),
|
||||
);
|
||||
|
||||
export const LOCATIONS: Record<number, string> = Object.fromEntries(
|
||||
LOCATION_OPTIONS.map((option) => [option.code, option.shortLabel]),
|
||||
);
|
||||
|
||||
const LOCATION_LANGUAGE: Record<number, string> = Object.fromEntries(
|
||||
LOCATION_OPTIONS.map((option) => [option.code, option.languageCode]),
|
||||
);
|
||||
|
||||
export function getLanguageCode(locationCode: number): string {
|
||||
return LOCATION_LANGUAGE[locationCode] ?? "en";
|
||||
}
|
||||
|
||||
export function isSupportedLocationCode(locationCode: number): boolean {
|
||||
return LOCATION_CODES.has(locationCode);
|
||||
}
|
||||
// Location data lives in shared/ so the server can route keyword research by
|
||||
// provider (Labs vs Google Ads). This shim keeps existing client imports.
|
||||
export {
|
||||
DEFAULT_LOCATION_CODE,
|
||||
LABS_LOCATION_OPTIONS,
|
||||
LOCATION_OPTIONS,
|
||||
LOCATIONS,
|
||||
getLanguageCode,
|
||||
isLabsLocationCode,
|
||||
isSupportedLocationCode,
|
||||
} from "@/shared/keyword-locations";
|
||||
|
||||
@ -40,6 +40,7 @@ export function KeywordResearchPage(input: Props) {
|
||||
loc: undefined,
|
||||
kLimit: undefined,
|
||||
mode: undefined,
|
||||
cs: undefined,
|
||||
});
|
||||
return;
|
||||
}
|
||||
@ -52,6 +53,7 @@ export function KeywordResearchPage(input: Props) {
|
||||
: tabInput.locationCode,
|
||||
kLimit: tabInput.resultLimit === 150 ? undefined : tabInput.resultLimit,
|
||||
mode: tabInput.mode === "auto" ? undefined : tabInput.mode,
|
||||
cs: tabInput.clickstream ? true : undefined,
|
||||
});
|
||||
},
|
||||
[setSearchParams],
|
||||
@ -67,8 +69,10 @@ export function KeywordResearchPage(input: Props) {
|
||||
locationCode: input.locationCode,
|
||||
resultLimit: input.resultLimit,
|
||||
mode: input.keywordMode,
|
||||
clickstream: input.clickstream,
|
||||
};
|
||||
}, [
|
||||
input.clickstream,
|
||||
input.keywordInput,
|
||||
input.keywordMode,
|
||||
input.locationCode,
|
||||
@ -108,6 +112,7 @@ export function KeywordResearchPage(input: Props) {
|
||||
locationCode: value.locationCode,
|
||||
resultLimit: value.resultLimit,
|
||||
mode: value.mode,
|
||||
clickstream: value.clickstream,
|
||||
}));
|
||||
|
||||
let activeInput: KeywordSearchTabInput | null = null;
|
||||
@ -135,6 +140,7 @@ export function KeywordResearchPage(input: Props) {
|
||||
locationCode: tab.input.locationCode,
|
||||
resultLimit: tab.input.resultLimit,
|
||||
mode: tab.input.mode,
|
||||
clickstream: tab.input.clickstream,
|
||||
},
|
||||
]
|
||||
: [],
|
||||
@ -152,6 +158,7 @@ export function KeywordResearchPage(input: Props) {
|
||||
hasExplicitLocationCode: true,
|
||||
resultLimit: activeTab.input.resultLimit,
|
||||
keywordMode: activeTab.input.mode,
|
||||
clickstream: activeTab.input.clickstream,
|
||||
getOpenKeywordTabs,
|
||||
keywordTabsLimit: searchTabs.limit,
|
||||
}
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
import { Search } from "lucide-react";
|
||||
import { Info, Search } from "lucide-react";
|
||||
import { getFieldError } from "@/client/lib/forms";
|
||||
import {
|
||||
isResultLimit,
|
||||
@ -8,7 +8,10 @@ import {
|
||||
MAX_KEYWORDS_PER_SUBMIT,
|
||||
RESULT_LIMITS,
|
||||
} from "@/client/features/keywords/keywordResearchTypes";
|
||||
import { LOCATION_OPTIONS } from "@/client/features/keywords/locations";
|
||||
import {
|
||||
LOCATION_OPTIONS,
|
||||
isLabsLocationCode,
|
||||
} from "@/client/features/keywords/locations";
|
||||
import type { KeywordResearchControllerState } from "./types";
|
||||
|
||||
type Props = {
|
||||
@ -139,6 +142,49 @@ export function KeywordResearchSearchBar({ controller }: Props) {
|
||||
) : null;
|
||||
}}
|
||||
</controlsForm.Field>
|
||||
<controlsForm.Field name="locationCode">
|
||||
{(locationField) =>
|
||||
isLabsLocationCode(locationField.state.value) ? (
|
||||
<controlsForm.Field name="clickstream">
|
||||
{(field) => (
|
||||
<div className="flex items-center gap-2">
|
||||
<label className="label cursor-pointer justify-start gap-2 p-0">
|
||||
<input
|
||||
type="checkbox"
|
||||
className="toggle toggle-sm toggle-primary"
|
||||
checked={field.state.value}
|
||||
onChange={(event) =>
|
||||
field.handleChange(event.target.checked)
|
||||
}
|
||||
/>
|
||||
<span className="text-sm font-medium text-base-content/80">
|
||||
Clickstream-refined volumes
|
||||
</span>
|
||||
</label>
|
||||
<div
|
||||
className="tooltip tooltip-right"
|
||||
data-tip="Google reports one combined search volume for similar keywords (e.g. 'seo tool' and 'seo tools'). Turn this on to estimate each keyword's own volume. Costs 2x the credits."
|
||||
>
|
||||
<Info className="size-3.5 text-base-content/50" />
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</controlsForm.Field>
|
||||
) : (
|
||||
<div
|
||||
className="flex items-start gap-2 rounded-lg border border-info/30 bg-info/10 px-3 py-2 text-sm text-base-content/80"
|
||||
role="status"
|
||||
>
|
||||
<Info className="mt-0.5 size-4 shrink-0 text-info" />
|
||||
<span>
|
||||
Keyword data for this country comes from Google Ads — search
|
||||
volume, CPC, and trends are available, but difficulty and
|
||||
intent are not.
|
||||
</span>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
</controlsForm.Field>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
|
||||
@ -65,12 +65,14 @@ export function buildKeywordSearchKey(params: {
|
||||
locationCode: number;
|
||||
resultLimit: ResultLimit;
|
||||
mode: KeywordMode;
|
||||
clickstream: boolean;
|
||||
}) {
|
||||
return [
|
||||
parseKeywordInput(params.keyword).join(""),
|
||||
params.locationCode,
|
||||
params.resultLimit,
|
||||
params.mode,
|
||||
params.clickstream ? "cs" : "",
|
||||
].join("|");
|
||||
}
|
||||
|
||||
|
||||
@ -34,6 +34,7 @@ type OpenKeywordTabInput = {
|
||||
locationCode: number;
|
||||
resultLimit: ResultLimit;
|
||||
mode: KeywordMode;
|
||||
clickstream: boolean;
|
||||
};
|
||||
|
||||
export type KeywordResearchControllerInput = {
|
||||
@ -43,6 +44,7 @@ export type KeywordResearchControllerInput = {
|
||||
hasExplicitLocationCode: boolean;
|
||||
resultLimit: ResultLimit;
|
||||
keywordMode: KeywordMode;
|
||||
clickstream: boolean;
|
||||
sortField: SortField;
|
||||
sortDir: SortDir;
|
||||
getOpenKeywordTabs?: () => readonly OpenKeywordTabInput[];
|
||||
@ -115,6 +117,7 @@ export function useKeywordResearchController(
|
||||
locationCode,
|
||||
resultLimit: input.resultLimit,
|
||||
mode: input.keywordMode,
|
||||
clickstream: input.clickstream,
|
||||
},
|
||||
addSearch,
|
||||
);
|
||||
@ -127,6 +130,7 @@ export function useKeywordResearchController(
|
||||
locationCode,
|
||||
resultLimit: input.resultLimit,
|
||||
mode: input.keywordMode,
|
||||
clickstream: input.clickstream,
|
||||
})
|
||||
: null;
|
||||
|
||||
|
||||
@ -9,6 +9,7 @@ import { Loader2, AlertCircle, X } from "lucide-react";
|
||||
import { toast } from "sonner";
|
||||
import { getDomainKeywordSuggestions } from "@/serverFunctions/domain";
|
||||
import { addTrackingKeywords } from "@/serverFunctions/rank-tracking";
|
||||
import { isLabsLocationCode } from "@/client/features/keywords/locations";
|
||||
import { getStandardErrorMessage } from "@/client/lib/error-messages";
|
||||
import {
|
||||
AppDataTable,
|
||||
@ -158,6 +159,9 @@ export function KeywordSuggestionStep({
|
||||
[],
|
||||
);
|
||||
|
||||
// Ranked-keyword suggestions are Labs-backed; countries served from Google
|
||||
// Ads keyword data (e.g. Iceland) have no ranking data to suggest from.
|
||||
const labsSupported = isLabsLocationCode(locationCode);
|
||||
const suggestionsQuery = useQuery({
|
||||
queryKey: [
|
||||
"domainKeywordSuggestions",
|
||||
@ -170,6 +174,7 @@ export function KeywordSuggestionStep({
|
||||
getDomainKeywordSuggestions({
|
||||
data: { projectId, domain, locationCode, languageCode },
|
||||
}),
|
||||
enabled: labsSupported,
|
||||
});
|
||||
|
||||
const data = suggestionsQuery.data ?? [];
|
||||
@ -238,6 +243,23 @@ export function KeywordSuggestionStep({
|
||||
</div>
|
||||
);
|
||||
|
||||
if (!labsSupported) {
|
||||
return (
|
||||
<>
|
||||
{sectionHeader("Add keywords manually")}
|
||||
<div className="flex flex-col items-center justify-center gap-3 py-16">
|
||||
<p className="text-xs text-base-content/50">
|
||||
Ranked-keyword suggestions aren't available for this country.
|
||||
Continue and add the keywords you want to track manually.
|
||||
</p>
|
||||
<button className="btn btn-primary btn-sm mt-2" onClick={onClose}>
|
||||
Continue
|
||||
</button>
|
||||
</div>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
// Loading state
|
||||
if (suggestionsQuery.isLoading) {
|
||||
return (
|
||||
|
||||
@ -220,6 +220,7 @@ function getSearchTabQueryConfig(
|
||||
locationCode: input.locationCode,
|
||||
resultLimit: input.resultLimit,
|
||||
mode: input.mode,
|
||||
clickstream: input.clickstream,
|
||||
});
|
||||
|
||||
return {
|
||||
|
||||
@ -23,6 +23,7 @@ export type KeywordSearchTabInput = {
|
||||
locationCode: number;
|
||||
resultLimit: ResultLimit;
|
||||
mode: KeywordMode;
|
||||
clickstream: boolean;
|
||||
};
|
||||
|
||||
export type SearchTabInput =
|
||||
|
||||
@ -77,6 +77,8 @@ function parseTabInput(value: unknown): SearchTabInput | null {
|
||||
locationCode: value.locationCode,
|
||||
resultLimit: value.resultLimit,
|
||||
mode: value.mode,
|
||||
// Tabs persisted before the clickstream toggle existed default to off.
|
||||
clickstream: value.clickstream === true,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@ -47,6 +47,7 @@ function KeywordResearchPageRoute() {
|
||||
hasExplicitLocationCode={search.loc != null}
|
||||
resultLimit={isResultLimit(resultLimit) ? resultLimit : 150}
|
||||
keywordMode={normalizeKeywordMode(keywordMode)}
|
||||
clickstream={search.cs ?? false}
|
||||
sortField={normalizeSortField(sortField)}
|
||||
sortDir={normalizeSortDir(sortDir)}
|
||||
/>
|
||||
|
||||
@ -0,0 +1,67 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
|
||||
vi.mock("@/server/lib/dataforseo", () => ({
|
||||
createDataforseoClient: vi.fn(),
|
||||
}));
|
||||
|
||||
import {
|
||||
KeywordsDataGoogleAdsKeywordsForKeywordsLiveResultInfo,
|
||||
MonthlySearchesInfo,
|
||||
} from "dataforseo-client";
|
||||
import { mapAdsKeywordItems } from "./research-data";
|
||||
|
||||
const adsItem = (
|
||||
data: ConstructorParameters<
|
||||
typeof KeywordsDataGoogleAdsKeywordsForKeywordsLiveResultInfo
|
||||
>[0],
|
||||
) => new KeywordsDataGoogleAdsKeywordsForKeywordsLiveResultInfo(data);
|
||||
|
||||
describe("mapAdsKeywordItems", () => {
|
||||
it("maps Google Ads items to research rows without KD/intent", () => {
|
||||
const rows = mapAdsKeywordItems([
|
||||
adsItem({
|
||||
keyword: "Hotel Reykjavik",
|
||||
search_volume: 1300,
|
||||
cpc: 2.54,
|
||||
competition: "HIGH",
|
||||
competition_index: 42,
|
||||
monthly_searches: [
|
||||
new MonthlySearchesInfo({
|
||||
year: 2026,
|
||||
month: 5,
|
||||
search_volume: 1300,
|
||||
}),
|
||||
],
|
||||
}),
|
||||
]);
|
||||
|
||||
expect(rows).toEqual([
|
||||
{
|
||||
keyword: "hotel reykjavik",
|
||||
searchVolume: 1300,
|
||||
trend: [{ year: 2026, month: 5, searchVolume: 1300 }],
|
||||
cpc: 2.54,
|
||||
competition: 0.42,
|
||||
keywordDifficulty: null,
|
||||
intent: "unknown",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("dedupes case-variant keywords and skips empty ones", () => {
|
||||
const rows = mapAdsKeywordItems([
|
||||
adsItem({ keyword: "northern lights tour", search_volume: 320 }),
|
||||
adsItem({ keyword: "Northern Lights Tour", search_volume: 320 }),
|
||||
adsItem({ keyword: undefined }),
|
||||
]);
|
||||
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0]).toMatchObject({
|
||||
keyword: "northern lights tour",
|
||||
searchVolume: 320,
|
||||
competition: null,
|
||||
cpc: null,
|
||||
trend: [],
|
||||
});
|
||||
});
|
||||
});
|
||||
@ -1,4 +1,7 @@
|
||||
import { type LabsKeywordDataItem } from "@/server/lib/dataforseo";
|
||||
import {
|
||||
type AdsKeywordIdeaItem,
|
||||
type LabsKeywordDataItem,
|
||||
} from "@/server/lib/dataforseo";
|
||||
import type { BillingCustomerContext } from "@/server/billing/subscription";
|
||||
import { createDataforseoClient } from "@/server/lib/dataforseo";
|
||||
import {
|
||||
@ -14,6 +17,7 @@ type FetchResearchRowsParams = {
|
||||
languageCode: string;
|
||||
resultLimit: number;
|
||||
source: KeywordSource;
|
||||
includeClickstreamData?: boolean;
|
||||
};
|
||||
|
||||
function mapKeywordDataItems(items: LabsKeywordDataItem[]): EnrichedKeyword[] {
|
||||
@ -28,6 +32,8 @@ function mapKeywordDataItems(items: LabsKeywordDataItem[]): EnrichedKeyword[] {
|
||||
if (seen.has(normalized)) continue;
|
||||
seen.add(normalized);
|
||||
|
||||
// The clickstream-normalized block only exists when the caller opted into
|
||||
// clickstream data (it doubles the request cost); prefer it when present.
|
||||
const keywordInfo = item.keyword_info_normalized_with_clickstream
|
||||
?.search_volume
|
||||
? item.keyword_info_normalized_with_clickstream
|
||||
@ -51,6 +57,59 @@ function mapKeywordDataItems(items: LabsKeywordDataItem[]): EnrichedKeyword[] {
|
||||
return rows;
|
||||
}
|
||||
|
||||
/**
|
||||
* Google Ads items carry volume / CPC / paid competition but no keyword
|
||||
* difficulty or search intent (those are Labs-only).
|
||||
*/
|
||||
export function mapAdsKeywordItems(
|
||||
items: AdsKeywordIdeaItem[],
|
||||
): EnrichedKeyword[] {
|
||||
const rows: EnrichedKeyword[] = [];
|
||||
const seen = new Set<string>();
|
||||
|
||||
for (const item of items) {
|
||||
const keyword = item.keyword;
|
||||
if (!keyword) continue;
|
||||
|
||||
const normalized = normalizeKeyword(keyword);
|
||||
if (seen.has(normalized)) continue;
|
||||
seen.add(normalized);
|
||||
|
||||
rows.push({
|
||||
keyword: normalized,
|
||||
searchVolume: item.search_volume ?? null,
|
||||
trend: (item.monthly_searches ?? []).map((entry) => ({
|
||||
year: entry.year ?? 0,
|
||||
month: entry.month ?? 0,
|
||||
searchVolume: entry.search_volume ?? 0,
|
||||
})),
|
||||
cpc: item.cpc ?? null,
|
||||
competition:
|
||||
item.competition_index != null ? item.competition_index / 100 : null,
|
||||
keywordDifficulty: null,
|
||||
intent: "unknown",
|
||||
});
|
||||
}
|
||||
|
||||
return rows;
|
||||
}
|
||||
|
||||
/** Research rows for countries DataForSEO Labs doesn't support. */
|
||||
export async function fetchGoogleAdsResearchRows(
|
||||
params: Omit<FetchResearchRowsParams, "source">,
|
||||
billingCustomer: BillingCustomerContext,
|
||||
): Promise<EnrichedKeyword[]> {
|
||||
const dataforseo = createDataforseoClient(billingCustomer);
|
||||
return mapAdsKeywordItems(
|
||||
await dataforseo.keywords.adsIdeas({
|
||||
keyword: params.seedKeyword,
|
||||
locationCode: params.locationCode,
|
||||
languageCode: params.languageCode,
|
||||
limit: params.resultLimit,
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
async function fetchRelatedRows(
|
||||
params: Omit<FetchResearchRowsParams, "source">,
|
||||
dataforseo: ReturnType<typeof createDataforseoClient>,
|
||||
@ -61,6 +120,7 @@ async function fetchRelatedRows(
|
||||
languageCode: params.languageCode,
|
||||
limit: params.resultLimit,
|
||||
depth: 3,
|
||||
includeClickstreamData: params.includeClickstreamData,
|
||||
});
|
||||
|
||||
// Related items wrap the keyword payload one level deeper; unwrap and reuse
|
||||
@ -89,6 +149,7 @@ export async function fetchResearchRowsBySource(
|
||||
locationCode: params.locationCode,
|
||||
languageCode: params.languageCode,
|
||||
limit: params.resultLimit,
|
||||
includeClickstreamData: params.includeClickstreamData,
|
||||
}),
|
||||
);
|
||||
}
|
||||
@ -99,6 +160,7 @@ export async function fetchResearchRowsBySource(
|
||||
locationCode: params.locationCode,
|
||||
languageCode: params.languageCode,
|
||||
limit: params.resultLimit,
|
||||
includeClickstreamData: params.includeClickstreamData,
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
@ -10,8 +10,12 @@ import { KeywordResearchRepository } from "@/server/features/keywords/repositori
|
||||
import type { KeywordResearchRow } from "@/types/keywords";
|
||||
import type { ResearchKeywordsInput } from "@/types/schemas/keywords";
|
||||
import { z } from "zod";
|
||||
import { getKeywordDataProvider } from "@/shared/keyword-locations";
|
||||
import { type EnrichedKeyword, normalizeKeyword } from "./helpers";
|
||||
import { fetchResearchRowsBySource } from "./research-data";
|
||||
import {
|
||||
fetchGoogleAdsResearchRows,
|
||||
fetchResearchRowsBySource,
|
||||
} from "./research-data";
|
||||
import {
|
||||
AUTO_KEYWORD_SOURCES,
|
||||
MIN_NON_SEED_FOR_AUTO,
|
||||
@ -19,10 +23,11 @@ import {
|
||||
hasSufficientCoverage,
|
||||
type KeywordMode,
|
||||
type KeywordSource,
|
||||
type ResearchSource,
|
||||
} from "./selection";
|
||||
|
||||
type SourceAttempt = {
|
||||
source: KeywordSource;
|
||||
source: ResearchSource;
|
||||
rowCount: number;
|
||||
nonSeedCount: number;
|
||||
};
|
||||
@ -35,7 +40,7 @@ type ResearchDiagnostics = {
|
||||
|
||||
type ResearchResult = {
|
||||
rows: KeywordResearchRow[];
|
||||
source: KeywordSource;
|
||||
source: ResearchSource;
|
||||
usedFallback: boolean;
|
||||
diagnostics: ResearchDiagnostics;
|
||||
};
|
||||
@ -65,14 +70,14 @@ const cachedKeywordRowSchema = z.object({
|
||||
});
|
||||
|
||||
const sourceAttemptSchema = z.object({
|
||||
source: z.enum(["related", "suggestions", "ideas"]),
|
||||
source: z.enum(["related", "suggestions", "ideas", "google_ads"]),
|
||||
rowCount: z.number(),
|
||||
nonSeedCount: z.number(),
|
||||
});
|
||||
|
||||
const cachedResultSchema = z.object({
|
||||
rows: z.array(cachedKeywordRowSchema),
|
||||
source: z.enum(["related", "suggestions", "ideas"]),
|
||||
source: z.enum(["related", "suggestions", "ideas", "google_ads"]),
|
||||
usedFallback: z.boolean(),
|
||||
diagnostics: z.object({
|
||||
requestedMode: z.enum(["auto", "related", "suggestions", "ideas"]),
|
||||
@ -81,7 +86,9 @@ const cachedResultSchema = z.object({
|
||||
}),
|
||||
});
|
||||
|
||||
const CACHE_VERSION = 2;
|
||||
// v3: research volumes are no longer clickstream-refined, and Google-Ads-only
|
||||
// locations route to keywords_for_keywords.
|
||||
const CACHE_VERSION = 3;
|
||||
|
||||
async function fetchRowsFromSource(
|
||||
source: KeywordSource,
|
||||
@ -96,6 +103,7 @@ async function fetchRowsFromSource(
|
||||
locationCode: input.locationCode,
|
||||
languageCode: input.languageCode,
|
||||
resultLimit: input.resultLimit,
|
||||
includeClickstreamData: input.clickstream,
|
||||
},
|
||||
billingCustomer,
|
||||
);
|
||||
@ -161,6 +169,39 @@ async function fetchAutoRows(
|
||||
};
|
||||
}
|
||||
|
||||
async function fetchGoogleAdsRows(
|
||||
input: ResearchKeywordsInput,
|
||||
seedKeyword: string,
|
||||
billingCustomer: BillingCustomerContext,
|
||||
): Promise<ResearchResult> {
|
||||
const rows = await fetchGoogleAdsResearchRows(
|
||||
{
|
||||
seedKeyword,
|
||||
locationCode: input.locationCode,
|
||||
languageCode: input.languageCode,
|
||||
resultLimit: input.resultLimit,
|
||||
},
|
||||
billingCustomer,
|
||||
);
|
||||
|
||||
return {
|
||||
rows,
|
||||
source: "google_ads",
|
||||
usedFallback: false,
|
||||
diagnostics: {
|
||||
requestedMode: "auto",
|
||||
threshold: MIN_NON_SEED_FOR_AUTO,
|
||||
sourceAttempts: [
|
||||
{
|
||||
source: "google_ads",
|
||||
rowCount: rows.length,
|
||||
nonSeedCount: countNonSeedKeywords(rows, seedKeyword),
|
||||
},
|
||||
],
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function fetchManualRows(
|
||||
mode: Exclude<KeywordMode, "auto">,
|
||||
input: ResearchKeywordsInput,
|
||||
@ -207,6 +248,7 @@ async function buildResearchCacheKey(
|
||||
resultLimit: input.resultLimit,
|
||||
mode,
|
||||
depth: 3,
|
||||
clickstream: input.clickstream,
|
||||
});
|
||||
}
|
||||
|
||||
@ -244,9 +286,17 @@ export async function research(
|
||||
}
|
||||
|
||||
const seedKeyword = uniqueKeywords[0];
|
||||
const mode = input.mode ?? "auto";
|
||||
const provider = getKeywordDataProvider(input.locationCode);
|
||||
// Labs source modes and clickstream refinement don't exist for
|
||||
// Google-Ads-served countries; collapse both so equivalent requests share
|
||||
// one cache entry.
|
||||
const effectiveInput: ResearchKeywordsInput =
|
||||
provider === "google_ads"
|
||||
? { ...input, mode: "auto", clickstream: false }
|
||||
: input;
|
||||
const mode = effectiveInput.mode ?? "auto";
|
||||
const cacheKey = await buildResearchCacheKey(
|
||||
input,
|
||||
effectiveInput,
|
||||
uniqueKeywords,
|
||||
mode,
|
||||
billingCustomer,
|
||||
@ -263,12 +313,19 @@ export async function research(
|
||||
}
|
||||
|
||||
const result =
|
||||
mode === "auto"
|
||||
? await fetchAutoRows(input, seedKeyword, billingCustomer)
|
||||
: await fetchManualRows(mode, input, seedKeyword, billingCustomer);
|
||||
provider === "google_ads"
|
||||
? await fetchGoogleAdsRows(effectiveInput, seedKeyword, billingCustomer)
|
||||
: mode === "auto"
|
||||
? await fetchAutoRows(effectiveInput, seedKeyword, billingCustomer)
|
||||
: await fetchManualRows(
|
||||
mode,
|
||||
effectiveInput,
|
||||
seedKeyword,
|
||||
billingCustomer,
|
||||
);
|
||||
|
||||
await setCached(cacheKey, result, CACHE_TTL.researchResult);
|
||||
persistRows(input, result.rows);
|
||||
persistRows(effectiveInput, result.rows);
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
@ -2,6 +2,11 @@ import type { EnrichedKeyword } from "./helpers";
|
||||
|
||||
export type KeywordSource = "related" | "suggestions" | "ideas";
|
||||
export type KeywordMode = "auto" | KeywordSource;
|
||||
/**
|
||||
* Where research rows actually came from. "google_ads" is not requestable as
|
||||
* a mode; it's the automatic source for countries Labs doesn't support.
|
||||
*/
|
||||
export type ResearchSource = KeywordSource | "google_ads";
|
||||
|
||||
export const AUTO_KEYWORD_SOURCES: KeywordSource[] = [
|
||||
"related",
|
||||
|
||||
@ -1,6 +1,7 @@
|
||||
import { env } from "cloudflare:workers";
|
||||
import type { BillingCustomerContext } from "@/server/billing/subscription";
|
||||
import { createDataforseoClient } from "@/server/lib/dataforseo";
|
||||
import { getKeywordDataProvider } from "@/shared/keyword-locations";
|
||||
import { RankTrackingRepository } from "@/server/features/rank-tracking/repositories/RankTrackingRepository";
|
||||
import { AppError } from "@/server/lib/errors";
|
||||
import type {
|
||||
@ -252,13 +253,17 @@ async function refreshKeywordMetrics(
|
||||
const now = new Date().toISOString();
|
||||
let updated = 0;
|
||||
|
||||
// Countries Labs doesn't cover get volume/CPC from Google Ads (no KD).
|
||||
const useGoogleAds =
|
||||
getKeywordDataProvider(config.locationCode) === "google_ads";
|
||||
|
||||
for (let i = 0; i < keywords.length; i += KEYWORD_OVERVIEW_BATCH_SIZE) {
|
||||
const batch = keywords.slice(i, i + KEYWORD_OVERVIEW_BATCH_SIZE);
|
||||
const items = await client.labs.keywordOverview({
|
||||
const request = {
|
||||
keywords: batch.map((kw) => kw.keyword),
|
||||
locationCode: config.locationCode,
|
||||
languageCode: config.languageCode,
|
||||
});
|
||||
};
|
||||
|
||||
// Build a lookup by lowercase keyword
|
||||
const metricsMap = new Map<
|
||||
@ -269,14 +274,30 @@ async function refreshKeywordMetrics(
|
||||
cpc: number | null;
|
||||
}
|
||||
>();
|
||||
for (const item of items) {
|
||||
if (useGoogleAds) {
|
||||
const adsItems = await client.keywords.adsSearchVolume({
|
||||
...request,
|
||||
creditFeature: "rank_tracking",
|
||||
});
|
||||
for (const item of adsItems) {
|
||||
if (!item.keyword) continue;
|
||||
metricsMap.set(item.keyword.toLowerCase(), {
|
||||
searchVolume: item.search_volume ?? null,
|
||||
keywordDifficulty: null,
|
||||
cpc: item.cpc ?? null,
|
||||
});
|
||||
}
|
||||
} else {
|
||||
for (const item of await client.labs.keywordOverview(request)) {
|
||||
if (!item.keyword) continue;
|
||||
metricsMap.set(item.keyword.toLowerCase(), {
|
||||
searchVolume: item.keyword_info?.search_volume ?? null,
|
||||
keywordDifficulty: item.keyword_properties?.keyword_difficulty ?? null,
|
||||
keywordDifficulty:
|
||||
item.keyword_properties?.keyword_difficulty ?? null,
|
||||
cpc: item.keyword_info?.cpc ?? null,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const updates = batch
|
||||
.map((kw) => {
|
||||
|
||||
@ -33,6 +33,10 @@ import {
|
||||
fetchRelevantPages,
|
||||
fetchSerpCompetitors,
|
||||
} from "@/server/lib/dataforseo/labs";
|
||||
import {
|
||||
fetchAdsKeywordIdeas,
|
||||
fetchAdsSearchVolume,
|
||||
} from "@/server/lib/dataforseo/google-ads";
|
||||
import {
|
||||
fetchLiveSerp,
|
||||
fetchLocalSerp,
|
||||
@ -101,6 +105,9 @@ export function createDataforseoClient(customer: BillingCustomerContext) {
|
||||
related: meter(customer, fetchRelatedKeywords),
|
||||
suggestions: meter(customer, fetchKeywordSuggestions),
|
||||
ideas: meter(customer, fetchKeywordIdeas),
|
||||
// Google Ads endpoints for countries Labs doesn't support.
|
||||
adsIdeas: meter(customer, fetchAdsKeywordIdeas),
|
||||
adsSearchVolume: meter(customer, fetchAdsSearchVolume),
|
||||
},
|
||||
domain: {
|
||||
rankOverview: meter(customer, fetchDomainRankOverview),
|
||||
|
||||
@ -3,6 +3,7 @@ import {
|
||||
BacklinksApi,
|
||||
BusinessDataApi,
|
||||
DataforseoLabsApi,
|
||||
KeywordsDataApi,
|
||||
OnPageApi,
|
||||
SerpApi,
|
||||
} from "dataforseo-client";
|
||||
@ -119,6 +120,7 @@ function http(classify?: DataforseoErrorClassifier) {
|
||||
// Per-section API factories. Each is created per-request so the auth secret is
|
||||
// read lazily (it lives in the Worker env, not in module scope).
|
||||
export const labsApi = () => new DataforseoLabsApi(API_BASE, http());
|
||||
export const keywordsDataApi = () => new KeywordsDataApi(API_BASE, http());
|
||||
export const serpApi = () => new SerpApi(API_BASE, http());
|
||||
export const businessDataApi = () => new BusinessDataApi(API_BASE, http());
|
||||
export const onPageApi = () => new OnPageApi(API_BASE, http());
|
||||
|
||||
69
src/server/lib/dataforseo/google-ads.ts
Normal file
69
src/server/lib/dataforseo/google-ads.ts
Normal file
@ -0,0 +1,69 @@
|
||||
import {
|
||||
KeywordsDataGoogleAdsKeywordsForKeywordsLiveRequestInfo,
|
||||
KeywordsDataGoogleAdsSearchVolumeLiveRequestInfo,
|
||||
type KeywordsDataGoogleAdsKeywordsForKeywordsLiveResultInfo,
|
||||
type KeywordsDataGoogleAdsSearchVolumeLiveResultInfo,
|
||||
} from "dataforseo-client";
|
||||
import { keywordsDataApi } from "@/server/lib/dataforseo/core";
|
||||
import {
|
||||
assertOk,
|
||||
buildTaskBilling,
|
||||
type DataforseoApiResponse,
|
||||
} from "@/server/lib/dataforseo/envelope";
|
||||
|
||||
// Google Ads keyword data for countries DataForSEO Labs doesn't cover (see
|
||||
// specs/0004-keyword-data-source-routing.md). Flat-priced per request; items
|
||||
// carry volume / CPC / competition but no keyword difficulty or intent.
|
||||
export type AdsKeywordItem = KeywordsDataGoogleAdsSearchVolumeLiveResultInfo;
|
||||
export type AdsKeywordIdeaItem =
|
||||
KeywordsDataGoogleAdsKeywordsForKeywordsLiveResultInfo;
|
||||
|
||||
type KeywordsDataResult<T> = { result?: T[] };
|
||||
|
||||
function taskItems<T>(task: KeywordsDataResult<T>): T[] {
|
||||
// keywords_data tasks return keyword items directly in `result` (no nested
|
||||
// `items` wrapper like Labs).
|
||||
return task.result ?? [];
|
||||
}
|
||||
|
||||
export async function fetchAdsSearchVolume(input: {
|
||||
keywords: string[];
|
||||
locationCode: number;
|
||||
languageCode: string;
|
||||
}): Promise<DataforseoApiResponse<AdsKeywordItem[]>> {
|
||||
const response = await keywordsDataApi().googleAdsSearchVolumeLive([
|
||||
new KeywordsDataGoogleAdsSearchVolumeLiveRequestInfo({
|
||||
keywords: input.keywords,
|
||||
location_code: input.locationCode,
|
||||
language_code: input.languageCode,
|
||||
}),
|
||||
]);
|
||||
const task = assertOk(response);
|
||||
return {
|
||||
data: taskItems(task),
|
||||
billing: buildTaskBilling(task),
|
||||
};
|
||||
}
|
||||
|
||||
export async function fetchAdsKeywordIdeas(input: {
|
||||
keyword: string;
|
||||
locationCode: number;
|
||||
languageCode: string;
|
||||
limit: number;
|
||||
}): Promise<DataforseoApiResponse<AdsKeywordIdeaItem[]>> {
|
||||
const response = await keywordsDataApi().googleAdsKeywordsForKeywordsLive([
|
||||
new KeywordsDataGoogleAdsKeywordsForKeywordsLiveRequestInfo({
|
||||
keywords: [input.keyword],
|
||||
location_code: input.locationCode,
|
||||
language_code: input.languageCode,
|
||||
sort_by: "search_volume",
|
||||
}),
|
||||
]);
|
||||
const task = assertOk(response);
|
||||
// The endpoint has no limit parameter (it can return thousands of
|
||||
// suggestions for one flat fee); truncate to what the caller asked for.
|
||||
return {
|
||||
data: taskItems(task).slice(0, input.limit),
|
||||
billing: buildTaskBilling(task),
|
||||
};
|
||||
}
|
||||
@ -12,6 +12,11 @@ export {
|
||||
type KeywordOverviewItem,
|
||||
} from "@/server/lib/dataforseo/labs";
|
||||
|
||||
export {
|
||||
type AdsKeywordItem,
|
||||
type AdsKeywordIdeaItem,
|
||||
} from "@/server/lib/dataforseo/google-ads";
|
||||
|
||||
export {
|
||||
type SerpLiveItem,
|
||||
type RankCheckResult,
|
||||
|
||||
@ -102,6 +102,7 @@ export async function fetchRelatedKeywords(input: {
|
||||
languageCode: string;
|
||||
limit: number;
|
||||
depth?: number;
|
||||
includeClickstreamData?: boolean;
|
||||
}): Promise<DataforseoApiResponse<RelatedKeywordItem[]>> {
|
||||
const response = await labsApi().googleRelatedKeywordsLive([
|
||||
new DataforseoLabsGoogleRelatedKeywordsLiveRequestInfo({
|
||||
@ -110,7 +111,9 @@ export async function fetchRelatedKeywords(input: {
|
||||
language_code: input.languageCode,
|
||||
limit: input.limit,
|
||||
depth: input.depth ?? 3,
|
||||
include_clickstream_data: true,
|
||||
// Clickstream-refined volumes DOUBLE the request cost, so they are
|
||||
// opt-in — see specs/0004-keyword-data-source-routing.md.
|
||||
include_clickstream_data: input.includeClickstreamData ?? false,
|
||||
include_serp_info: false,
|
||||
}),
|
||||
]);
|
||||
@ -126,6 +129,7 @@ export async function fetchKeywordSuggestions(input: {
|
||||
locationCode: number;
|
||||
languageCode: string;
|
||||
limit: number;
|
||||
includeClickstreamData?: boolean;
|
||||
}): Promise<DataforseoApiResponse<LabsKeywordDataItem[]>> {
|
||||
const response = await labsApi().googleKeywordSuggestionsLive([
|
||||
new DataforseoLabsGoogleKeywordSuggestionsLiveRequestInfo({
|
||||
@ -133,7 +137,7 @@ export async function fetchKeywordSuggestions(input: {
|
||||
location_code: input.locationCode,
|
||||
language_code: input.languageCode,
|
||||
limit: input.limit,
|
||||
include_clickstream_data: true,
|
||||
include_clickstream_data: input.includeClickstreamData ?? false,
|
||||
include_serp_info: false,
|
||||
include_seed_keyword: true,
|
||||
ignore_synonyms: false,
|
||||
@ -152,6 +156,7 @@ export async function fetchKeywordIdeas(input: {
|
||||
locationCode: number;
|
||||
languageCode: string;
|
||||
limit: number;
|
||||
includeClickstreamData?: boolean;
|
||||
}): Promise<DataforseoApiResponse<LabsKeywordDataItem[]>> {
|
||||
const response = await labsApi().googleKeywordIdeasLive([
|
||||
new DataforseoLabsGoogleKeywordIdeasLiveRequestInfo({
|
||||
@ -159,7 +164,7 @@ export async function fetchKeywordIdeas(input: {
|
||||
location_code: input.locationCode,
|
||||
language_code: input.languageCode,
|
||||
limit: input.limit,
|
||||
include_clickstream_data: true,
|
||||
include_clickstream_data: input.includeClickstreamData ?? false,
|
||||
include_serp_info: false,
|
||||
ignore_synonyms: false,
|
||||
closely_variants: false,
|
||||
@ -274,12 +279,14 @@ export async function fetchKeywordOverview(input: {
|
||||
keywords: string[];
|
||||
locationCode: number;
|
||||
languageCode: string;
|
||||
includeClickstreamData?: boolean;
|
||||
}): Promise<DataforseoApiResponse<KeywordOverviewItem[]>> {
|
||||
const response = await labsApi().googleKeywordOverviewLive([
|
||||
new DataforseoLabsGoogleKeywordOverviewLiveRequestInfo({
|
||||
keywords: input.keywords,
|
||||
location_code: input.locationCode,
|
||||
language_code: input.languageCode,
|
||||
include_clickstream_data: input.includeClickstreamData ?? false,
|
||||
}),
|
||||
]);
|
||||
const task = assertOk(response);
|
||||
|
||||
@ -1,4 +1,6 @@
|
||||
import { z } from "zod";
|
||||
import { AppError } from "@/server/lib/errors";
|
||||
import { getKeywordDataProvider } from "@/shared/keyword-locations";
|
||||
|
||||
export const DEFAULT_LOCATION_CODE = 2840;
|
||||
export const DEFAULT_LANGUAGE_CODE = "en";
|
||||
@ -15,9 +17,22 @@ export const locationCodeSchema = z
|
||||
.int()
|
||||
.positive()
|
||||
.describe(
|
||||
"DataForSEO location code. Defaults to 2840 (United States). See dataforseo.com/help-center/locations.",
|
||||
"DataForSEO location code. Defaults to 2840 (United States). See dataforseo.com/help-center/locations. Some countries (e.g. Iceland, 2352) are served from Google Ads data: keyword volume/CPC/trends work, but keyword difficulty, search intent, and domain analytics are unavailable.",
|
||||
);
|
||||
|
||||
/**
|
||||
* Guards Labs-backed tools (domain analytics) against locations we serve
|
||||
* from Google Ads keyword data only.
|
||||
*/
|
||||
export function assertLabsLocationCode(locationCode: number | undefined) {
|
||||
if (locationCode != null && getKeywordDataProvider(locationCode) !== "labs") {
|
||||
throw new AppError(
|
||||
"VALIDATION_ERROR",
|
||||
"Domain analytics is not available for this country. Keyword research and rank tracking work; domain-level data is limited to DataForSEO Labs locations.",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
export const languageCodeSchema = z
|
||||
.string()
|
||||
.min(2)
|
||||
|
||||
@ -0,0 +1,157 @@
|
||||
import type { AuthInfo } from "@modelcontextprotocol/sdk/server/auth/types.js";
|
||||
import type { ToolExtra } from "@/server/mcp/context";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { z } from "zod";
|
||||
import { MCP_AUTH_CONTEXT_PROP } from "@/server/mcp/context";
|
||||
|
||||
const mocks = vi.hoisted(() => ({
|
||||
createDataforseoClient: vi.fn(),
|
||||
getProjectForOrganization: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock("cloudflare:workers", () => ({
|
||||
env: {},
|
||||
}));
|
||||
|
||||
vi.mock("@/server/lib/dataforseo", () => ({
|
||||
createDataforseoClient: mocks.createDataforseoClient,
|
||||
}));
|
||||
|
||||
vi.mock("@/server/features/projects/services/ProjectService", () => ({
|
||||
ProjectService: {
|
||||
getProjectForOrganization: mocks.getProjectForOrganization,
|
||||
},
|
||||
}));
|
||||
|
||||
const authContext = {
|
||||
userId: "user_123",
|
||||
userEmail: "alice@example.com",
|
||||
organizationId: "org_123",
|
||||
clientId: "client_123",
|
||||
scopes: ["mcp"],
|
||||
audience: "https://open-seo.test/mcp",
|
||||
subject: "user_123",
|
||||
baseUrl: "https://open-seo.test",
|
||||
};
|
||||
|
||||
const toolExtra: ToolExtra = {
|
||||
signal: new AbortController().signal,
|
||||
requestId: 1,
|
||||
sendNotification: vi.fn(),
|
||||
sendRequest: vi.fn(),
|
||||
authInfo: {
|
||||
token: "token",
|
||||
clientId: "client_123",
|
||||
scopes: ["mcp"],
|
||||
resource: new URL("https://open-seo.test/mcp"),
|
||||
extra: { [MCP_AUTH_CONTEXT_PROP]: authContext },
|
||||
} satisfies AuthInfo,
|
||||
};
|
||||
|
||||
describe("get_keyword_metrics for Google-Ads-only locations", () => {
|
||||
beforeEach(() => {
|
||||
vi.resetModules();
|
||||
mocks.createDataforseoClient.mockReset();
|
||||
mocks.getProjectForOrganization.mockReset();
|
||||
mocks.getProjectForOrganization.mockResolvedValue({ id: "project_1" });
|
||||
});
|
||||
|
||||
it("serves Iceland from adsSearchVolume without KD/intent", async () => {
|
||||
const keywordOverview = vi.fn();
|
||||
const adsSearchVolume = vi.fn().mockResolvedValue([
|
||||
{
|
||||
keyword: "hotel reykjavik",
|
||||
search_volume: 1300,
|
||||
cpc: 2.54,
|
||||
competition: "HIGH",
|
||||
competition_index: 42,
|
||||
monthly_searches: [{ year: 2026, month: 5, search_volume: 1300 }],
|
||||
},
|
||||
]);
|
||||
|
||||
mocks.createDataforseoClient.mockReturnValue({
|
||||
labs: { keywordOverview },
|
||||
keywords: { adsSearchVolume },
|
||||
});
|
||||
const { getKeywordMetricsTool } =
|
||||
await import("./dataforseo-research-tools");
|
||||
|
||||
const result = await getKeywordMetricsTool.handler(
|
||||
{
|
||||
projectId: "project_1",
|
||||
keywords: ["hotel reykjavik"],
|
||||
// Iceland is not supported by DataForSEO Labs.
|
||||
locationCode: 2352,
|
||||
languageCode: "is",
|
||||
},
|
||||
toolExtra,
|
||||
);
|
||||
|
||||
expect(keywordOverview).not.toHaveBeenCalled();
|
||||
expect(adsSearchVolume).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
keywords: ["hotel reykjavik"],
|
||||
locationCode: 2352,
|
||||
languageCode: "is",
|
||||
creditFeature: "keyword_research",
|
||||
}),
|
||||
);
|
||||
const rows = z
|
||||
.object({ keywords: z.array(z.record(z.string(), z.unknown())) })
|
||||
.passthrough()
|
||||
.parse(result.structuredContent).keywords;
|
||||
expect(rows[0]).toMatchObject({
|
||||
keyword: "hotel reykjavik",
|
||||
search_volume: 1300,
|
||||
keyword_difficulty: null,
|
||||
main_intent: null,
|
||||
cpc: 2.54,
|
||||
competition: 0.42,
|
||||
competition_level: "HIGH",
|
||||
});
|
||||
});
|
||||
|
||||
it("passes the clickstream opt-in to Labs and prefers refined volumes", async () => {
|
||||
const keywordOverview = vi.fn().mockResolvedValue([
|
||||
{
|
||||
keyword: "seo tools",
|
||||
keyword_info: {
|
||||
search_volume: 10000,
|
||||
monthly_searches: [{ year: 2026, month: 5, search_volume: 10000 }],
|
||||
},
|
||||
keyword_info_normalized_with_clickstream: {
|
||||
search_volume: 6400,
|
||||
monthly_searches: [{ year: 2026, month: 5, search_volume: 6400 }],
|
||||
},
|
||||
},
|
||||
]);
|
||||
|
||||
mocks.createDataforseoClient.mockReturnValue({
|
||||
labs: { keywordOverview },
|
||||
});
|
||||
const { getKeywordMetricsTool } =
|
||||
await import("./dataforseo-research-tools");
|
||||
|
||||
const result = await getKeywordMetricsTool.handler(
|
||||
{
|
||||
projectId: "project_1",
|
||||
keywords: ["seo tools"],
|
||||
includeClickstreamData: true,
|
||||
},
|
||||
toolExtra,
|
||||
);
|
||||
|
||||
expect(keywordOverview).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ includeClickstreamData: true }),
|
||||
);
|
||||
const rows = z
|
||||
.object({ keywords: z.array(z.record(z.string(), z.unknown())) })
|
||||
.passthrough()
|
||||
.parse(result.structuredContent).keywords;
|
||||
expect(rows[0]).toMatchObject({
|
||||
keyword: "seo tools",
|
||||
search_volume: 6400,
|
||||
monthly_searches: [{ year: 2026, month: 5, search_volume: 6400 }],
|
||||
});
|
||||
});
|
||||
});
|
||||
@ -2,8 +2,10 @@
|
||||
import { z } from "zod";
|
||||
import {
|
||||
createDataforseoClient,
|
||||
type AdsKeywordItem,
|
||||
type KeywordOverviewItem,
|
||||
} from "@/server/lib/dataforseo";
|
||||
import { getKeywordDataProvider } from "@/shared/keyword-locations";
|
||||
import { buildProjectMeta } from "@/server/mcp/context";
|
||||
import { mcpResponse } from "@/server/mcp/formatters";
|
||||
import {
|
||||
@ -303,6 +305,12 @@ const getKeywordMetricsInputSchema = {
|
||||
.boolean()
|
||||
.optional()
|
||||
.describe("Include monthly search-volume trend rows. Defaults to true."),
|
||||
includeClickstreamData: z
|
||||
.boolean()
|
||||
.optional()
|
||||
.describe(
|
||||
"Refine search volumes with clickstream data, which disaggregates Google Ads' grouped close-variant volumes (plurals/misspellings). DOUBLES the credit cost of the call. Default false. No effect for countries served from Google Ads data.",
|
||||
),
|
||||
sortBy: keywordMetricsSortSchema
|
||||
.optional()
|
||||
.describe("Sort order for returned rows. Defaults to search_volume."),
|
||||
@ -457,20 +465,41 @@ function sortCompetitors(
|
||||
|
||||
function normalizeKeywordOverview(item: KeywordOverviewItem) {
|
||||
const info = item.keyword_info;
|
||||
// Only present when the caller opted into clickstream-refined volumes.
|
||||
const clickstreamInfo = item.keyword_info_normalized_with_clickstream;
|
||||
return {
|
||||
keyword: item.keyword,
|
||||
search_volume: info?.search_volume ?? null,
|
||||
search_volume:
|
||||
clickstreamInfo?.search_volume ?? info?.search_volume ?? null,
|
||||
keyword_difficulty: item.keyword_properties?.keyword_difficulty ?? null,
|
||||
main_intent: item.search_intent_info?.main_intent ?? null,
|
||||
cpc: info?.cpc ?? null,
|
||||
competition: info?.competition ?? null,
|
||||
competition_level: info?.competition_level ?? null,
|
||||
monthly_searches: info?.monthly_searches ?? null,
|
||||
monthly_searches:
|
||||
(clickstreamInfo?.search_volume
|
||||
? clickstreamInfo.monthly_searches
|
||||
: info?.monthly_searches) ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
type KeywordMetricRow = ReturnType<typeof normalizeKeywordOverview>;
|
||||
|
||||
// Google Ads items (countries Labs doesn't cover) have no difficulty/intent.
|
||||
function normalizeAdsKeyword(item: AdsKeywordItem): KeywordMetricRow {
|
||||
return {
|
||||
keyword: item.keyword,
|
||||
search_volume: item.search_volume ?? null,
|
||||
keyword_difficulty: null,
|
||||
main_intent: null,
|
||||
cpc: item.cpc ?? null,
|
||||
competition:
|
||||
item.competition_index != null ? item.competition_index / 100 : null,
|
||||
competition_level: item.competition ?? null,
|
||||
monthly_searches: item.monthly_searches ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
function sortKeywordMetricRows(
|
||||
rows: KeywordMetricRow[],
|
||||
sortBy: NonNullable<GetKeywordMetricsArgs["sortBy"]> = "search_volume",
|
||||
@ -722,7 +751,7 @@ export const getKeywordMetricsTool = {
|
||||
config: {
|
||||
title: "Get keyword metrics",
|
||||
description:
|
||||
"Hydrate up to 700 known keywords with search volume, keyword difficulty (KD), search intent, CPC, competition, and monthly trends in a single call. Use it to score candidate or known keywords — including Search Console striking-distance queries — by real demand and ranking difficulty. Charges credits.",
|
||||
"Hydrate up to 700 known keywords with search volume, keyword difficulty (KD), search intent, CPC, competition, and monthly trends in a single call. Use it to score candidate or known keywords — including Search Console striking-distance queries — by real demand and ranking difficulty. For countries served from Google Ads data (e.g. Iceland), KD and intent are null. Charges credits.",
|
||||
inputSchema: getKeywordMetricsInputSchema,
|
||||
outputSchema: {
|
||||
keywords: z.array(looseObjectOutputSchema),
|
||||
@ -736,14 +765,29 @@ export const getKeywordMetricsTool = {
|
||||
},
|
||||
handler: withMcpProjectAuth(async (args: GetKeywordMetricsArgs, context) => {
|
||||
const client = createDataforseoClient(context.billing);
|
||||
const items = await client.labs.keywordOverview({
|
||||
const locationCode = args.locationCode ?? DEFAULT_LOCATION_CODE;
|
||||
const languageCode = args.languageCode ?? DEFAULT_LANGUAGE_CODE;
|
||||
const normalized =
|
||||
getKeywordDataProvider(locationCode) === "google_ads"
|
||||
? (
|
||||
await client.keywords.adsSearchVolume({
|
||||
keywords: args.keywords,
|
||||
locationCode: args.locationCode ?? DEFAULT_LOCATION_CODE,
|
||||
languageCode: args.languageCode ?? DEFAULT_LANGUAGE_CODE,
|
||||
locationCode,
|
||||
languageCode,
|
||||
creditFeature: "keyword_research",
|
||||
});
|
||||
})
|
||||
).map(normalizeAdsKeyword)
|
||||
: (
|
||||
await client.labs.keywordOverview({
|
||||
keywords: args.keywords,
|
||||
locationCode,
|
||||
languageCode,
|
||||
includeClickstreamData: args.includeClickstreamData ?? false,
|
||||
creditFeature: "keyword_research",
|
||||
})
|
||||
).map(normalizeKeywordOverview);
|
||||
const rows = sortKeywordMetricRows(
|
||||
items.map(normalizeKeywordOverview),
|
||||
normalized,
|
||||
args.sortBy ?? "search_volume",
|
||||
).map((row) =>
|
||||
args.includeMonthlyTrends === false
|
||||
|
||||
@ -10,6 +10,7 @@ import { withMcpProjectAuth } from "@/server/mcp/project-auth";
|
||||
import {
|
||||
DEFAULT_LANGUAGE_CODE,
|
||||
DEFAULT_LOCATION_CODE,
|
||||
assertLabsLocationCode,
|
||||
languageCodeSchema,
|
||||
locationCodeSchema,
|
||||
projectIdSchema,
|
||||
@ -45,6 +46,7 @@ export const getDomainKeywordSuggestionsTool = {
|
||||
},
|
||||
},
|
||||
handler: withMcpProjectAuth(async (args: Args, context) => {
|
||||
assertLabsLocationCode(args.locationCode);
|
||||
const keywords = await DomainService.getSuggestedKeywords(
|
||||
{
|
||||
domain: args.domain,
|
||||
|
||||
@ -7,6 +7,7 @@ import { withMcpProjectAuth } from "@/server/mcp/project-auth";
|
||||
import {
|
||||
DEFAULT_LANGUAGE_CODE,
|
||||
DEFAULT_LOCATION_CODE,
|
||||
assertLabsLocationCode,
|
||||
languageCodeSchema,
|
||||
locationCodeSchema,
|
||||
projectIdSchema,
|
||||
@ -50,6 +51,7 @@ export const getDomainOverviewTool = {
|
||||
},
|
||||
},
|
||||
handler: withMcpProjectAuth(async (args: Args, context) => {
|
||||
assertLabsLocationCode(args.locationCode);
|
||||
const result = await DomainService.getOverview(
|
||||
{
|
||||
projectId: args.projectId,
|
||||
|
||||
@ -34,6 +34,12 @@ const inputSchema = {
|
||||
.union([z.literal(150), z.literal(300), z.literal(500)])
|
||||
.optional()
|
||||
.describe("Max keywords returned per seed. Defaults to 150."),
|
||||
includeClickstreamData: z
|
||||
.boolean()
|
||||
.optional()
|
||||
.describe(
|
||||
"Refine search volumes with clickstream data, which disaggregates Google Ads' grouped close-variant volumes (plurals/misspellings). DOUBLES the credit cost of each seed. Default false (standard Google-Ads-derived volumes). No effect for countries served from Google Ads data.",
|
||||
),
|
||||
} as const;
|
||||
|
||||
type Args = z.infer<z.ZodObject<typeof inputSchema>>;
|
||||
@ -43,7 +49,7 @@ export const researchKeywordsTool = {
|
||||
config: {
|
||||
title: "Research keywords (bulk)",
|
||||
description:
|
||||
"Research keyword data (search volume, difficulty, CPC, related ideas) for 1-5 seed keywords in one call. Charges credits per seed (~50-200 credits each, varies by source). Returns per-seed results — a single bad seed won't fail the batch.",
|
||||
"Research keyword data (search volume, difficulty, CPC, related ideas) for 1-5 seed keywords in one call. Charges credits per seed (~30-100 credits each, varies by source; flat ~96 for countries served from Google Ads data, where difficulty/intent are unavailable). Returns per-seed results — a single bad seed won't fail the batch.",
|
||||
inputSchema,
|
||||
outputSchema: {
|
||||
results: z.array(
|
||||
@ -87,6 +93,7 @@ export const researchKeywordsTool = {
|
||||
languageCode: item.languageCode ?? DEFAULT_LANGUAGE_CODE,
|
||||
resultLimit: args.resultLimit ?? 150,
|
||||
mode: "auto",
|
||||
clickstream: args.includeClickstreamData ?? false,
|
||||
},
|
||||
context.billing,
|
||||
);
|
||||
|
||||
49
src/shared/keyword-locations.test.ts
Normal file
49
src/shared/keyword-locations.test.ts
Normal file
@ -0,0 +1,49 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
LABS_LOCATION_OPTIONS,
|
||||
LOCATION_OPTIONS,
|
||||
getKeywordDataProvider,
|
||||
getLanguageCode,
|
||||
isLabsLocationCode,
|
||||
isSupportedLocationCode,
|
||||
} from "./keyword-locations";
|
||||
|
||||
describe("keyword locations", () => {
|
||||
it("routes Labs-supported countries to labs", () => {
|
||||
expect(getKeywordDataProvider(2840)).toBe("labs"); // US
|
||||
expect(getKeywordDataProvider(2826)).toBe("labs"); // UK
|
||||
});
|
||||
|
||||
it("routes Google-Ads-only countries to google_ads", () => {
|
||||
expect(getKeywordDataProvider(2352)).toBe("google_ads"); // Iceland
|
||||
expect(isSupportedLocationCode(2352)).toBe(true);
|
||||
expect(isLabsLocationCode(2352)).toBe(false);
|
||||
expect(getLanguageCode(2352)).toBe("is");
|
||||
});
|
||||
|
||||
it("falls back to labs for unknown codes (Labs rejects them upstream)", () => {
|
||||
expect(getKeywordDataProvider(999999)).toBe("labs");
|
||||
expect(isSupportedLocationCode(999999)).toBe(false);
|
||||
});
|
||||
|
||||
it("excludes every Google-Ads-only country from the Labs picker", () => {
|
||||
const adsOnly = LOCATION_OPTIONS.filter((option) => option.googleAdsOnly);
|
||||
expect(adsOnly.length).toBeGreaterThan(0);
|
||||
const labsCodes = new Set(
|
||||
LABS_LOCATION_OPTIONS.map((option) => option.code),
|
||||
);
|
||||
for (const option of adsOnly) {
|
||||
expect(labsCodes.has(option.code)).toBe(false);
|
||||
}
|
||||
expect(LABS_LOCATION_OPTIONS.length + adsOnly.length).toBe(
|
||||
LOCATION_OPTIONS.length,
|
||||
);
|
||||
});
|
||||
|
||||
it("keeps the picker sorted alphabetically with unique codes", () => {
|
||||
const labels = LOCATION_OPTIONS.map((option) => option.label);
|
||||
expect(labels).toEqual(labels.toSorted((a, b) => a.localeCompare(b)));
|
||||
const codes = LOCATION_OPTIONS.map((option) => option.code);
|
||||
expect(new Set(codes).size).toBe(codes.length);
|
||||
});
|
||||
});
|
||||
538
src/shared/keyword-locations.ts
Normal file
538
src/shared/keyword-locations.ts
Normal file
@ -0,0 +1,538 @@
|
||||
/* eslint-disable max-lines -- country data table */
|
||||
/**
|
||||
* Supported keyword-data countries and their data provider.
|
||||
*
|
||||
* Default provider is DataForSEO Labs (94 countries; source:
|
||||
* https://api.dataforseo.com/v3/dataforseo_labs/locations_and_languages).
|
||||
* Countries Labs does not cover are marked `googleAdsOnly` and are served by
|
||||
* the DataForSEO Keywords Data API (Google Ads endpoints), which covers the
|
||||
* full Google geotarget list — see specs/0004-keyword-data-source-routing.md.
|
||||
* Google-Ads-only rows have no keyword difficulty or search intent.
|
||||
*
|
||||
* For countries with multiple Google-supported languages, we pick the
|
||||
* language with the largest keyword corpus (the primary search market)
|
||||
* as the default. The APIs accept a single location_code + language_code
|
||||
* pair per request, so we expose one entry per country. Language codes for
|
||||
* googleAdsOnly entries must exist in BOTH the Google Ads and SERP language
|
||||
* lists (rank tracking shares this picker and uses the SERP API).
|
||||
*
|
||||
* Entries are sorted alphabetically by country name; pick US as the
|
||||
* product-wide default via DEFAULT_LOCATION_CODE below.
|
||||
*/
|
||||
export const DEFAULT_LOCATION_CODE = 2840;
|
||||
|
||||
type KeywordDataProvider = "labs" | "google_ads";
|
||||
|
||||
type LocationOption = {
|
||||
code: number;
|
||||
label: string;
|
||||
shortLabel: string;
|
||||
languageCode: string;
|
||||
/** Set when DataForSEO Labs does not support this country. */
|
||||
googleAdsOnly?: true;
|
||||
};
|
||||
|
||||
export const LOCATION_OPTIONS: readonly LocationOption[] = [
|
||||
{ code: 2008, label: "Albania", shortLabel: "AL", languageCode: "sq" },
|
||||
{ code: 2012, label: "Algeria", shortLabel: "DZ", languageCode: "fr" },
|
||||
{
|
||||
code: 2020,
|
||||
label: "Andorra",
|
||||
shortLabel: "AD",
|
||||
languageCode: "ca",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2024, label: "Angola", shortLabel: "AO", languageCode: "pt" },
|
||||
{ code: 2032, label: "Argentina", shortLabel: "AR", languageCode: "es" },
|
||||
{ code: 2051, label: "Armenia", shortLabel: "AM", languageCode: "hy" },
|
||||
{ code: 2036, label: "Australia", shortLabel: "AU", languageCode: "en" },
|
||||
{ code: 2040, label: "Austria", shortLabel: "AT", languageCode: "de" },
|
||||
{ code: 2031, label: "Azerbaijan", shortLabel: "AZ", languageCode: "az" },
|
||||
{
|
||||
code: 2044,
|
||||
label: "Bahamas",
|
||||
shortLabel: "BS",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2048, label: "Bahrain", shortLabel: "BH", languageCode: "ar" },
|
||||
{ code: 2050, label: "Bangladesh", shortLabel: "BD", languageCode: "bn" },
|
||||
{
|
||||
code: 2052,
|
||||
label: "Barbados",
|
||||
shortLabel: "BB",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2056, label: "Belgium", shortLabel: "BE", languageCode: "nl" },
|
||||
{
|
||||
code: 2084,
|
||||
label: "Belize",
|
||||
shortLabel: "BZ",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2068, label: "Bolivia", shortLabel: "BO", languageCode: "es" },
|
||||
{
|
||||
code: 2070,
|
||||
label: "Bosnia and Herzegovina",
|
||||
shortLabel: "BA",
|
||||
languageCode: "bs",
|
||||
},
|
||||
{
|
||||
code: 2072,
|
||||
label: "Botswana",
|
||||
shortLabel: "BW",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2076, label: "Brazil", shortLabel: "BR", languageCode: "pt" },
|
||||
{
|
||||
code: 2096,
|
||||
label: "Brunei",
|
||||
shortLabel: "BN",
|
||||
languageCode: "ms",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2100, label: "Bulgaria", shortLabel: "BG", languageCode: "bg" },
|
||||
{ code: 2854, label: "Burkina Faso", shortLabel: "BF", languageCode: "fr" },
|
||||
{ code: 2116, label: "Cambodia", shortLabel: "KH", languageCode: "en" },
|
||||
{ code: 2120, label: "Cameroon", shortLabel: "CM", languageCode: "fr" },
|
||||
{ code: 2124, label: "Canada", shortLabel: "CA", languageCode: "en" },
|
||||
{ code: 2152, label: "Chile", shortLabel: "CL", languageCode: "es" },
|
||||
{ code: 2170, label: "Colombia", shortLabel: "CO", languageCode: "es" },
|
||||
{ code: 2188, label: "Costa Rica", shortLabel: "CR", languageCode: "es" },
|
||||
{ code: 2384, label: "Cote d'Ivoire", shortLabel: "CI", languageCode: "fr" },
|
||||
{ code: 2191, label: "Croatia", shortLabel: "HR", languageCode: "hr" },
|
||||
{ code: 2196, label: "Cyprus", shortLabel: "CY", languageCode: "el" },
|
||||
{ code: 2203, label: "Czechia", shortLabel: "CZ", languageCode: "cs" },
|
||||
{ code: 2208, label: "Denmark", shortLabel: "DK", languageCode: "da" },
|
||||
{
|
||||
code: 2214,
|
||||
label: "Dominican Republic",
|
||||
shortLabel: "DO",
|
||||
languageCode: "es",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2218, label: "Ecuador", shortLabel: "EC", languageCode: "es" },
|
||||
{ code: 2818, label: "Egypt", shortLabel: "EG", languageCode: "ar" },
|
||||
{ code: 2222, label: "El Salvador", shortLabel: "SV", languageCode: "es" },
|
||||
{ code: 2233, label: "Estonia", shortLabel: "EE", languageCode: "et" },
|
||||
{
|
||||
code: 2231,
|
||||
label: "Ethiopia",
|
||||
shortLabel: "ET",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2242,
|
||||
label: "Fiji",
|
||||
shortLabel: "FJ",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2246, label: "Finland", shortLabel: "FI", languageCode: "fi" },
|
||||
{ code: 2250, label: "France", shortLabel: "FR", languageCode: "fr" },
|
||||
{
|
||||
code: 2268,
|
||||
label: "Georgia",
|
||||
shortLabel: "GE",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2276, label: "Germany", shortLabel: "DE", languageCode: "de" },
|
||||
{ code: 2288, label: "Ghana", shortLabel: "GH", languageCode: "en" },
|
||||
{ code: 2300, label: "Greece", shortLabel: "GR", languageCode: "el" },
|
||||
{ code: 2320, label: "Guatemala", shortLabel: "GT", languageCode: "es" },
|
||||
{
|
||||
code: 2831,
|
||||
label: "Guernsey",
|
||||
shortLabel: "GG",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2328,
|
||||
label: "Guyana",
|
||||
shortLabel: "GY",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2332,
|
||||
label: "Haiti",
|
||||
shortLabel: "HT",
|
||||
languageCode: "fr",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2340,
|
||||
label: "Honduras",
|
||||
shortLabel: "HN",
|
||||
languageCode: "es",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2344, label: "Hong Kong", shortLabel: "HK", languageCode: "zh-TW" },
|
||||
{ code: 2348, label: "Hungary", shortLabel: "HU", languageCode: "hu" },
|
||||
{
|
||||
code: 2352,
|
||||
label: "Iceland",
|
||||
shortLabel: "IS",
|
||||
languageCode: "is",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2356, label: "India", shortLabel: "IN", languageCode: "en" },
|
||||
{ code: 2360, label: "Indonesia", shortLabel: "ID", languageCode: "id" },
|
||||
{
|
||||
code: 2368,
|
||||
label: "Iraq",
|
||||
shortLabel: "IQ",
|
||||
languageCode: "ar",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2372, label: "Ireland", shortLabel: "IE", languageCode: "en" },
|
||||
{
|
||||
code: 2833,
|
||||
label: "Isle of Man",
|
||||
shortLabel: "IM",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2376, label: "Israel", shortLabel: "IL", languageCode: "he" },
|
||||
{ code: 2380, label: "Italy", shortLabel: "IT", languageCode: "it" },
|
||||
{
|
||||
code: 2388,
|
||||
label: "Jamaica",
|
||||
shortLabel: "JM",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2392, label: "Japan", shortLabel: "JP", languageCode: "ja" },
|
||||
{
|
||||
code: 2832,
|
||||
label: "Jersey",
|
||||
shortLabel: "JE",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2400, label: "Jordan", shortLabel: "JO", languageCode: "ar" },
|
||||
{ code: 2398, label: "Kazakhstan", shortLabel: "KZ", languageCode: "ru" },
|
||||
{ code: 2404, label: "Kenya", shortLabel: "KE", languageCode: "en" },
|
||||
{
|
||||
code: 2414,
|
||||
label: "Kuwait",
|
||||
shortLabel: "KW",
|
||||
languageCode: "ar",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2417,
|
||||
label: "Kyrgyzstan",
|
||||
shortLabel: "KG",
|
||||
languageCode: "ru",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2418,
|
||||
label: "Laos",
|
||||
shortLabel: "LA",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2428, label: "Latvia", shortLabel: "LV", languageCode: "lv" },
|
||||
{
|
||||
code: 2422,
|
||||
label: "Lebanon",
|
||||
shortLabel: "LB",
|
||||
languageCode: "ar",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2438,
|
||||
label: "Liechtenstein",
|
||||
shortLabel: "LI",
|
||||
languageCode: "de",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2440, label: "Lithuania", shortLabel: "LT", languageCode: "lt" },
|
||||
{
|
||||
code: 2442,
|
||||
label: "Luxembourg",
|
||||
shortLabel: "LU",
|
||||
languageCode: "fr",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2450,
|
||||
label: "Madagascar",
|
||||
shortLabel: "MG",
|
||||
languageCode: "fr",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2454,
|
||||
label: "Malawi",
|
||||
shortLabel: "MW",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2458, label: "Malaysia", shortLabel: "MY", languageCode: "en" },
|
||||
{
|
||||
code: 2462,
|
||||
label: "Maldives",
|
||||
shortLabel: "MV",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2470, label: "Malta", shortLabel: "MT", languageCode: "en" },
|
||||
{
|
||||
code: 2480,
|
||||
label: "Mauritius",
|
||||
shortLabel: "MU",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2484, label: "Mexico", shortLabel: "MX", languageCode: "es" },
|
||||
{ code: 2498, label: "Moldova", shortLabel: "MD", languageCode: "ro" },
|
||||
{ code: 2492, label: "Monaco", shortLabel: "MC", languageCode: "fr" },
|
||||
{
|
||||
code: 2496,
|
||||
label: "Mongolia",
|
||||
shortLabel: "MN",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2499,
|
||||
label: "Montenegro",
|
||||
shortLabel: "ME",
|
||||
languageCode: "sr",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2504, label: "Morocco", shortLabel: "MA", languageCode: "ar" },
|
||||
{
|
||||
code: 2508,
|
||||
label: "Mozambique",
|
||||
shortLabel: "MZ",
|
||||
languageCode: "pt",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2104,
|
||||
label: "Myanmar (Burma)",
|
||||
shortLabel: "MM",
|
||||
languageCode: "en",
|
||||
},
|
||||
{
|
||||
code: 2516,
|
||||
label: "Namibia",
|
||||
shortLabel: "NA",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2524,
|
||||
label: "Nepal",
|
||||
shortLabel: "NP",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2528, label: "Netherlands", shortLabel: "NL", languageCode: "nl" },
|
||||
{ code: 2554, label: "New Zealand", shortLabel: "NZ", languageCode: "en" },
|
||||
{ code: 2558, label: "Nicaragua", shortLabel: "NI", languageCode: "es" },
|
||||
{ code: 2566, label: "Nigeria", shortLabel: "NG", languageCode: "en" },
|
||||
{
|
||||
code: 2807,
|
||||
label: "North Macedonia",
|
||||
shortLabel: "MK",
|
||||
languageCode: "mk",
|
||||
},
|
||||
{ code: 2578, label: "Norway", shortLabel: "NO", languageCode: "nb" },
|
||||
{
|
||||
code: 2512,
|
||||
label: "Oman",
|
||||
shortLabel: "OM",
|
||||
languageCode: "ar",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2586, label: "Pakistan", shortLabel: "PK", languageCode: "en" },
|
||||
{ code: 2591, label: "Panama", shortLabel: "PA", languageCode: "es" },
|
||||
{
|
||||
code: 2598,
|
||||
label: "Papua New Guinea",
|
||||
shortLabel: "PG",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2600, label: "Paraguay", shortLabel: "PY", languageCode: "es" },
|
||||
{ code: 2604, label: "Peru", shortLabel: "PE", languageCode: "es" },
|
||||
{ code: 2608, label: "Philippines", shortLabel: "PH", languageCode: "en" },
|
||||
{ code: 2616, label: "Poland", shortLabel: "PL", languageCode: "pl" },
|
||||
{ code: 2620, label: "Portugal", shortLabel: "PT", languageCode: "pt" },
|
||||
{
|
||||
code: 2634,
|
||||
label: "Qatar",
|
||||
shortLabel: "QA",
|
||||
languageCode: "ar",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2642, label: "Romania", shortLabel: "RO", languageCode: "ro" },
|
||||
{
|
||||
code: 2646,
|
||||
label: "Rwanda",
|
||||
shortLabel: "RW",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2674,
|
||||
label: "San Marino",
|
||||
shortLabel: "SM",
|
||||
languageCode: "it",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2682, label: "Saudi Arabia", shortLabel: "SA", languageCode: "ar" },
|
||||
{ code: 2686, label: "Senegal", shortLabel: "SN", languageCode: "fr" },
|
||||
{ code: 2688, label: "Serbia", shortLabel: "RS", languageCode: "sr" },
|
||||
{ code: 2702, label: "Singapore", shortLabel: "SG", languageCode: "en" },
|
||||
{ code: 2703, label: "Slovakia", shortLabel: "SK", languageCode: "sk" },
|
||||
{ code: 2705, label: "Slovenia", shortLabel: "SI", languageCode: "sl" },
|
||||
{ code: 2710, label: "South Africa", shortLabel: "ZA", languageCode: "en" },
|
||||
{ code: 2410, label: "South Korea", shortLabel: "KR", languageCode: "ko" },
|
||||
{ code: 2724, label: "Spain", shortLabel: "ES", languageCode: "es" },
|
||||
{ code: 2144, label: "Sri Lanka", shortLabel: "LK", languageCode: "en" },
|
||||
{
|
||||
code: 2740,
|
||||
label: "Suriname",
|
||||
shortLabel: "SR",
|
||||
languageCode: "nl",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2752, label: "Sweden", shortLabel: "SE", languageCode: "sv" },
|
||||
{ code: 2756, label: "Switzerland", shortLabel: "CH", languageCode: "de" },
|
||||
{ code: 2158, label: "Taiwan", shortLabel: "TW", languageCode: "zh-TW" },
|
||||
{
|
||||
code: 2762,
|
||||
label: "Tajikistan",
|
||||
shortLabel: "TJ",
|
||||
languageCode: "ru",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2834,
|
||||
label: "Tanzania",
|
||||
shortLabel: "TZ",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2764, label: "Thailand", shortLabel: "TH", languageCode: "th" },
|
||||
{
|
||||
code: 2780,
|
||||
label: "Trinidad and Tobago",
|
||||
shortLabel: "TT",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2788, label: "Tunisia", shortLabel: "TN", languageCode: "ar" },
|
||||
{ code: 2792, label: "Turkiye", shortLabel: "TR", languageCode: "tr" },
|
||||
{
|
||||
code: 2795,
|
||||
label: "Turkmenistan",
|
||||
shortLabel: "TM",
|
||||
languageCode: "ru",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2800,
|
||||
label: "Uganda",
|
||||
shortLabel: "UG",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2804, label: "Ukraine", shortLabel: "UA", languageCode: "uk" },
|
||||
{
|
||||
code: 2784,
|
||||
label: "United Arab Emirates",
|
||||
shortLabel: "AE",
|
||||
languageCode: "en",
|
||||
},
|
||||
{
|
||||
code: 2826,
|
||||
label: "United Kingdom",
|
||||
shortLabel: "UK",
|
||||
languageCode: "en",
|
||||
},
|
||||
{ code: 2840, label: "United States", shortLabel: "US", languageCode: "en" },
|
||||
{ code: 2858, label: "Uruguay", shortLabel: "UY", languageCode: "es" },
|
||||
{
|
||||
code: 2860,
|
||||
label: "Uzbekistan",
|
||||
shortLabel: "UZ",
|
||||
languageCode: "ru",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{ code: 2862, label: "Venezuela", shortLabel: "VE", languageCode: "es" },
|
||||
{ code: 2704, label: "Vietnam", shortLabel: "VN", languageCode: "vi" },
|
||||
{
|
||||
code: 2894,
|
||||
label: "Zambia",
|
||||
shortLabel: "ZM",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
{
|
||||
code: 2716,
|
||||
label: "Zimbabwe",
|
||||
shortLabel: "ZW",
|
||||
languageCode: "en",
|
||||
googleAdsOnly: true,
|
||||
},
|
||||
] as const;
|
||||
|
||||
/** Countries usable by DataForSEO Labs features (domain overview etc.). */
|
||||
export const LABS_LOCATION_OPTIONS = LOCATION_OPTIONS.filter(
|
||||
(option) => !option.googleAdsOnly,
|
||||
);
|
||||
|
||||
const LOCATION_CODES = new Set<number>(
|
||||
LOCATION_OPTIONS.map((option) => option.code),
|
||||
);
|
||||
|
||||
const LABS_LOCATION_CODES = new Set<number>(
|
||||
LABS_LOCATION_OPTIONS.map((option) => option.code),
|
||||
);
|
||||
|
||||
export const LOCATIONS: Record<number, string> = Object.fromEntries(
|
||||
LOCATION_OPTIONS.map((option) => [option.code, option.shortLabel]),
|
||||
);
|
||||
|
||||
const LOCATION_LANGUAGE: Record<number, string> = Object.fromEntries(
|
||||
LOCATION_OPTIONS.map((option) => [option.code, option.languageCode]),
|
||||
);
|
||||
|
||||
export function getLanguageCode(locationCode: number): string {
|
||||
return LOCATION_LANGUAGE[locationCode] ?? "en";
|
||||
}
|
||||
|
||||
export function isSupportedLocationCode(locationCode: number): boolean {
|
||||
return LOCATION_CODES.has(locationCode);
|
||||
}
|
||||
|
||||
export function isLabsLocationCode(locationCode: number): boolean {
|
||||
return LABS_LOCATION_CODES.has(locationCode);
|
||||
}
|
||||
|
||||
/**
|
||||
* Which DataForSEO API serves keyword data for this location. Unknown codes
|
||||
* fall back to Labs so behavior for arbitrary codes is unchanged (Labs
|
||||
* rejects unsupported locations with its own error).
|
||||
*/
|
||||
export function getKeywordDataProvider(
|
||||
locationCode: number,
|
||||
): KeywordDataProvider {
|
||||
return LOCATION_CODES.has(locationCode) &&
|
||||
!LABS_LOCATION_CODES.has(locationCode)
|
||||
? "google_ads"
|
||||
: "labs";
|
||||
}
|
||||
@ -51,7 +51,7 @@ export const domainField = z
|
||||
}
|
||||
});
|
||||
|
||||
const booleanSearchParamSchema = z
|
||||
export const booleanSearchParamSchema = z
|
||||
.union([z.boolean(), z.enum(["true", "false"])])
|
||||
.transform((value) => value === true || value === "true");
|
||||
|
||||
|
||||
@ -1,5 +1,6 @@
|
||||
import { z } from "zod";
|
||||
import { TAG_COLOR_KEYS } from "@/shared/tag-colors";
|
||||
import { booleanSearchParamSchema } from "@/types/schemas/domain";
|
||||
|
||||
const savedKeywordTagSchema = z.string().trim().min(1).max(64);
|
||||
const tagColorSchema = z.enum(TAG_COLOR_KEYS);
|
||||
@ -26,6 +27,8 @@ export const researchKeywordsSchema = z.object({
|
||||
.enum(["auto", "related", "suggestions", "ideas"])
|
||||
.optional()
|
||||
.default("auto"),
|
||||
// Clickstream-refined volumes double the DataForSEO request cost; opt-in.
|
||||
clickstream: z.boolean().optional().default(false),
|
||||
});
|
||||
|
||||
export const saveKeywordsSchema = z
|
||||
@ -184,6 +187,7 @@ export const keywordsSearchSchema = z.object({
|
||||
loc: z.coerce.number().int().positive().optional(),
|
||||
kLimit: z.union([z.literal(150), z.literal(300), z.literal(500)]).optional(),
|
||||
mode: z.enum(keywordModes).optional(),
|
||||
cs: booleanSearchParamSchema.optional(),
|
||||
sort: z.enum(keywordSortFields).optional(),
|
||||
order: z.enum(sortDirs).optional(),
|
||||
minVol: z.string().optional(),
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user