OAuth, Passkeys, and Account Linking

OAuth, passkeys, and account linking solve one practical problem: the same person may prove identity in several ways, and your Next.js application must turn those proofs into one coherent local user account. OAuth lets a user authenticate through an external identity provider such as Google or GitHub. Passkeys let the user authenticate with a WebAuthn credential protected by their device, security key, or platform authenticator. Account linking decides when two proofs should attach to the same local user record.

The outcome is not merely a sign-in button. By the end of this lesson, you should be able to describe the protocol boundaries, model the database records, handle callbacks in App Router route handlers, verify passkey ceremonies on the server, and choose conservative linking rules that avoid account takeover. This fits the authentication section of the Next.js course because every protected Server Component, Server Action, and Route Handler depends on the identity object these flows create.

How the Pieces Fit

Your application should have one internal user identifier that never changes, even if the user later adds GitHub, removes Google, or registers a new passkey. External credentials attach to that user. A common relational model uses users, oauth_accounts, webauthn_credentials, and sessions. The OAuth table stores provider name, provider subject, tokens if needed, scopes, and expiry. The passkey table stores credential id, public key, sign count or backup state metadata, transports, and the user id. The session table stores a server-issued session id or token binding the browser to the local user.

OAuth uses redirects. The browser leaves your site for the provider, the provider authenticates the user, and the browser returns to your callback URL with a short-lived authorization code. The server exchanges that code for tokens, validates the provider response, reads the provider subject, and maps it to a local user. The important identifier is not the email address; it is the provider subject within that provider. Emails can change, be unverified, or be reused by different identity systems.

Passkeys use a challenge-response ceremony. For registration, your server creates a random challenge and options containing relying party information, user information, and acceptable authenticator behavior. The browser calls WebAuthn APIs, the authenticator creates a key pair, and the server stores the credential public key after verifying the attestation response. For authentication, the server sends a new challenge, the authenticator signs it with the private key, and the server verifies the signature with the stored public key. The private key never reaches your application.

Account linking is the policy layer. Linking is safe when the currently authenticated user intentionally adds another credential, or when you have a provider guarantee strong enough for your threat model. Linking is risky when it silently merges accounts because two providers report the same email. A defensible default is explicit linking only: the user signs in, visits security settings, starts the new provider or passkey flow, and confirms the result before the credential is attached.

API Anatomy

In Next.js App Router, OAuth callbacks and WebAuthn endpoints usually live in app/api route handlers because they receive browser requests, set cookies, and return redirects or JSON. Server Components then read the session through a server-only helper. Client Components are limited to button clicks and browser-only APIs such as navigator.credentials.create and navigator.credentials.get.

Three values deserve careful names. state is an OAuth nonce stored before redirect and checked on callback to prevent cross-site request forgery. code_verifier and code_challenge are PKCE values that bind the callback to the browser session that initiated OAuth. challenge is the WebAuthn random value that prevents replay of old registration or authentication responses. These values should be short lived, single use, and stored server side or in sealed, integrity-protected cookies.

Example 1: Normalize Provider Identity

The first step is turning provider profile data into a stable internal shape. This example intentionally refuses an OAuth profile without a provider subject and records whether the email was verified. Expected behavior: a valid GitHub profile becomes a provider key such as github:12345; a missing subject throws before account lookup.

type OAuthProfile = {
  provider: "github" | "google";
  subject: string;
  email?: string;
  emailVerified?: boolean;
};

export function normalizeOAuthProfile(profile: OAuthProfile) {
  if (!profile.subject) {
    throw new Error("OAuth provider did not return a stable subject");
  }

  return {
    providerAccountKey: `${profile.provider}:${profile.subject}`,
    provider: profile.provider,
    providerSubject: profile.subject,
    email: profile.email?.toLowerCase() ?? null,
    emailVerified: profile.emailVerified === true,
  };
}

This function is small, but it encodes a major design choice: local lookup is based on provider plus subject, not display name or email. Email can still help with onboarding messages or a manual linking prompt, but it should not be the only proof that two accounts are the same person.

Example 2: OAuth Callback Flow

The callback handler checks state, exchanges the code, normalizes the provider identity, and either finds or creates the local user. Expected behavior: if the stored state and query state differ, the handler stops with a 400 response; if they match, the handler creates a session and redirects into the app.

import { NextRequest, NextResponse } from "next/server";

export async function GET(request: NextRequest) {
  const url = new URL(request.url);
  const code = url.searchParams.get("code");
  const returnedState = url.searchParams.get("state");
  const expectedState = request.cookies.get("oauth_state")?.value;

  if (!code || !returnedState || returnedState !== expectedState) {
    return NextResponse.json({ error: "invalid_oauth_callback" }, { status: 400 });
  }

  const profile = await exchangeCodeForProfile(code);
  const account = normalizeOAuthProfile(profile);
  const user = await findOrCreateUserForOAuthAccount(account);
  const response = NextResponse.redirect(new URL("/dashboard", request.url));

  await attachSessionCookie(response, user.id);
  response.cookies.delete("oauth_state");
  return response;
}

The handler belongs on the server because it handles secrets, cookies, token exchange, and database writes. A Client Component may render the sign-in button, but the redirect URL generation and callback validation should be server controlled.

Example 3: Register a Passkey

Passkey registration is a two-request flow. The first request creates options and stores the challenge. The browser then asks the authenticator to create a credential. The second request verifies the authenticator response and stores the credential public key. Expected behavior: the stored challenge is consumed once; replaying the same registration response fails because the challenge is no longer valid.

export async function registerPasskey(userId: string, response: unknown) {
  const pending = await loadPendingWebAuthnChallenge(userId, "registration");
  if (!pending) {
    throw new Error("No active passkey registration challenge");
  }

  const verification = await verifyRegistrationResponse({
    response,
    expectedChallenge: pending.challenge,
    expectedOrigin: process.env.AUTH_ORIGIN!,
    expectedRPID: process.env.AUTH_RP_ID!,
  });

  if (!verification.verified || !verification.registrationInfo) {
    throw new Error("Passkey registration could not be verified");
  }

  await saveWebAuthnCredential({
    userId,
    credentialId: verification.registrationInfo.credentialID,
    publicKey: verification.registrationInfo.credentialPublicKey,
    counter: verification.registrationInfo.counter,
  });
  await deletePendingWebAuthnChallenge(pending.id);
}

Libraries such as SimpleWebAuthn handle the binary parsing and signature verification details. Your responsibility is to pass the correct relying party id and origin, persist the right fields, and make the challenge single use. In development, localhost has special browser treatment, but production passkeys require a secure origin and a relying party id that matches your domain.

Example 4: Explicit Account Linking

Linking should be a transaction because two requests may try to attach the same provider account at the same time. Expected behavior: the unique constraint prevents one provider account from belonging to two users; the transaction either links the account to the current user or reports that it is already linked elsewhere.

BEGIN;

SELECT id
FROM users
WHERE id = $1
FOR UPDATE;

INSERT INTO oauth_accounts (user_id, provider, provider_subject)
VALUES ($1, $2, $3)
ON CONFLICT (provider, provider_subject) DO NOTHING;

COMMIT;

The application must inspect whether the insert actually happened. If it did not, show a neutral message such as This sign-in method is already linked. Do not reveal the email or owner of the other account.

Design Choices and Trade-offs

OAuth is excellent for reducing password handling and letting users reuse trusted identity providers. Its trade-off is dependency on provider availability, provider configuration, redirect correctness, and token lifecycle management. Request the smallest scope set you need. If you only need sign-in, avoid storing long-lived access tokens. If you need API access, encrypt refresh tokens at rest and design a revocation path.

Passkeys resist phishing because the authenticator signs challenges for a specific relying party id. They also remove password reset flows for users who have multiple synced credentials or backup methods. The trade-off is ecosystem complexity: some users share devices, some authenticators are not synced, and account recovery must be designed before enforcement. Offer more than one enrolled credential for high-value accounts and require reauthentication before sensitive changes.

Automatic linking by verified email is convenient but dangerous when provider semantics differ. One provider’s verified email may not mean the same thing as another provider’s enterprise-managed identity. Explicit linking costs an extra click, but it gives you a clear user intent signal and a safer audit trail.

Failure Modes and Troubleshooting

OAuth callback returns 400. The symptom is a failed redirect after provider sign-in. The likely cause is mismatched state, a missing cookie, an incorrect callback URL, or a SameSite cookie setting that prevented the state cookie from returning. Diagnose by logging a bounded correlation id, callback URL, and whether the state cookie existed, without logging tokens. Correct by registering the exact callback URL with the provider and setting the initiating cookie from the same site that receives the callback.

OAuth creates duplicate users. The symptom is that the same person sees a new empty dashboard after using a different provider. The cause is usually lookup by email without a linking policy, or lookup by provider subject without offering a linking path. Diagnose by checking oauth_accounts for the provider subject and comparing session history. Correct by adding explicit linking from account settings and by showing an existing-account prompt when a verified email matches but no link exists.

Passkey registration works locally but fails in production. The symptom is a browser WebAuthn error or a server verification failure. The cause is commonly an origin or relying party id mismatch, such as using www.example.com in the browser but example.com in server expectations. Diagnose by comparing the request origin, configured AUTH_ORIGIN, and AUTH_RP_ID. Correct the environment values and clear stale pending challenges.

Account linking can be stolen through session fixation. The symptom may be a provider account unexpectedly linked to the wrong user. The cause is starting a link flow before reauthentication or failing to bind the OAuth state to the current local user. Diagnose by auditing link events for session id, user id, provider, and time. Correct by requiring a fresh session before linking and storing the intended user id with the link challenge.

Security and Reliability Implications

Keep tokens, WebAuthn challenges, and session secrets out of Client Components. Mark helpers that read secrets as server-only. Use unique constraints on (provider, provider_subject) and credential_id. Rate limit callback and WebAuthn verification endpoints because they perform database work and cryptographic verification. Log outcomes as categories such as state_mismatch, unknown_provider_account, or credential_verification_failed, not raw credentials.

Session issuance is the final authority. After any OAuth or passkey success, create a fresh session id, set secure cookie attributes, and rotate the session when the user links credentials or changes security settings. For cached Next.js pages, avoid reading identity in places that accidentally become static. Protected pages should read cookies or headers on the server and render dynamically when user-specific data is present.

Hands-on Lab

Prerequisites: a Next.js App Router project, a database with unique constraints for OAuth accounts and passkey credentials, an OAuth application from a provider, and a WebAuthn verification library. Use a local HTTPS tunnel or localhost for development, and set AUTH_ORIGIN, AUTH_RP_ID, provider client id, and provider secret in environment variables.

  1. Create tables for users, OAuth accounts, WebAuthn credentials, sessions, and pending challenges. Add unique indexes on provider identity and credential id.
  2. Add a server route that begins OAuth by creating state and PKCE values, storing them, and redirecting to the provider authorization URL.
  3. Add the callback route from the example, replacing placeholder helpers with your provider exchange and database functions.
  4. Add passkey registration options and verification endpoints. Store the challenge before calling browser WebAuthn APIs and delete it after successful verification.
  5. Add an account settings page where a signed-in user can link a new OAuth provider or register a passkey. Require recent authentication before either action.
  6. Verify OAuth sign-in creates one local user, OAuth sign-in repeats into the same user, passkey authentication signs into that user, and trying to link an already linked provider shows a neutral refusal.
  7. Cleanup by deleting test provider links, passkey credentials, sessions, and pending challenges. Revoke test OAuth tokens in the provider console if you requested API scopes.

Assessment

  1. Design a database constraint set that prevents one OAuth provider account from being linked to two users while still allowing one user to link several providers. Explain what each constraint prevents.
  2. A user signs in with GitHub, then later with Google using the same verified email and sees a new account. What linking options would you offer, and what proof would you require before merging?
  3. Your passkey verification fails only after deploying behind a new domain. List the values you would inspect and explain how relying party id differs from origin.
  4. Write the refusal behavior for a provider account that is already linked to another user. What should the UI say, and what should the audit log store?
  5. Explain why a Client Component can start a passkey ceremony but should not decide whether the resulting credential is valid.

Summary

OAuth proves identity through a provider redirect and stable provider subject. Passkeys prove possession of a private key bound to your relying party. Account linking connects those proofs to one local user only when policy says the evidence is strong enough. In a Next.js application, keep protocol verification in route handlers, keep secrets server side, model credentials separately from users, and make linking explicit, transactional, and auditable.