Exploring the Intricacies of OTP Authentication in Next.js

Passwords aren’t great. They’re often weak, reused, and need to be stored indefinitely making them susceptible to attacks and leaks. have i been pwned?⁠ tells me passwords associated with just one of my email addresses have been leaked 18 times, and I have over 800 individual passwords stored in my password manager.

This is why magic links, SSO, and one-time passwords (OTPs) are becoming standard authentication methods. They provide better or additional security for your users.

Here we’re going to look at OTPs. One-time passwords are a great option for improving the security of your application. Let’s go through exactly what one-time passwords are, how they can improve the security of your application, the best practices for using them in authentication, and how you can implement OTPs in Next.js.

What is a one-time password?

OTP or One-Time Password authentication is a method in which a unique code is sent to a user's device and the user enters this code into an application to verify their identity. It’s valid for only one login session and usually time-limited.

There are two ways OTPs are implemented:

1. As the main factor for logging in a user. If you don’t want to use any username/password combinations in your authentication flow, you can send a one-time password for log in. You can also use it as an alternative to username/password. This is not particularly common, but can be seen on sites such as local social networking site Nextdoor.
2. As an additional step in multi-factor authentication (MFA). After a user has logged in with their username/password or another authentication provider (such as Google, Twitter, or GitHub), they are then sent an OTP to a registered email address or phone number as a secondary layer of security for applications. This second option is much more common.

You’d think that one-time passwords would be long and complicated like the suggested passwords from a password manager. But because they are transient in nature and rarely subject to dictionary or brute force attacks, they can be much simpler. OTPs can have different formats, but they usually consist of a series of alphanumeric characters or purely numeric characters. The length of OTPs can vary, but a common format is a 6 or 8-digit numeric code.

The choice between numeric and alphanumeric OTPs depends on the context and requirements. Numeric OTPs can be easier to enter, especially on mobile devices, which makes them a popular choice for SMS-based OTPs. However, alphanumeric OTPs provide a larger possible combination set for the same number of characters, which can be more secure against brute-force attacks.

There are six main steps in the OTP flow:

1. Trigger. The user initiates the OTP process. This could be a login attempt, a transaction verification, or any other scenario where identity needs to be confirmed.
2. OTP generation. The server generates an OTP. This is typically a random string or number that is time-limited.
3. OTP delivery. The generated OTP is sent to the user through a predetermined method. This could be an SMS to their phone, an email to their registered email address, or generated in a hardware token or software app, such as Google Authenticator.
4. User input. The user receives the OTP and enters it into the application.
5. OTP verification. The server then verifies the OTP. The server also verifies that the OTP is used within the allowed time period, and hasn't been used before.
6. Authentication: If the OTP is verified, the server authenticates the user and allows them to proceed. If the OTP is incorrect or expired, the server rejects the request.

Though this is only six steps, there are a lot of moving parts in OTP use. You need extra frontend UI design and logic to handle the OTP inputs; you need to integrate a delivery mechanism such as SMS, email, or an authenticator app; you need extra authentication logic to handle OTP errors; and you need the OTP generation and verification logic as well.

There are several ways to generate and verify an OTP:

• Time-Synchronized: In this method, the OTP is generated by applying a cryptographic hash function to a shared secret and the current time, typically measured in intervals of 30 seconds. The Google Authenticator app uses this method.
• Counter-Based: Here, the OTP is generated by hashing a shared secret with a counter which increments with each new OTP.
• Algorithm-Based: In this method, a mathematical algorithm is applied to the previous password to generate the new one.

If a time-based or counter-based OTP was used, the server repeats the same OTP generation process during verification and checks if the received OTP matches the one it generated.

These algorithms aren’t easy to implement. As you are dealing with an authentication factor, there are specific designs that you must use and RFCs that you must follow, such as this one for Time-Based OTPs⁠. This is the biggest challenge around building your own one-time password (or any authentication) system–implementation.

The security benefits (and challenges) of OTPs

You can quickly see the complexity involved in setting up OTP authentication. But doing so is worth it as they provide two key security benefits.

Risk mitigation

The mitigation of risk with OTPs comes from two different avenues.

The first is mitigating the risk associated with static passwords. Traditional static passwords, if stolen, provide ongoing access until they are changed. OTPs are dynamic and expire after a single use or after a short period of time, limiting the potential damage if they are intercepted or stolen. Another problem is users reusing the same password across multiple services. If one service is compromised, all accounts using the same password are at risk. OTPs eliminate this risk because they are unique for each login session.

The second is the reduction of attack vectors. Because OTPs are typically time-limited, it makes brute-force attacks infeasible. An attacker doesn't have the time to try all possible combinations before the password expires. They also help with phishing. Even if a user is tricked into entering their OTP into a phishing site, the attacker can't reuse that OTP to gain future access to the account.

Additionally, since an OTP is valid for only one login session or transaction, it cannot be reused, preventing replay attacks. In a replay attack, an attacker tries to reuse a password that was intercepted in a previous session. However, if there's a flaw in the system's design where the OTP doesn't expire immediately after use or isn't time-bound, there's a possibility for replay attacks.

Identity security and verification

OTPs also play a significant role in enhancing identity security and verification processes. They provide an additional layer of protection beyond traditional static passwords, aiding in the confirmation of a user's identity in several ways:

• OTPs are often used as part of a two-factor authentication process. In addition to a traditional username and password (something the user knows), the user must enter an OTP (something the user has, typically their mobile device). This confirms that the user has access to a specific device (like a phone) that is associated with the account, verifying the identity of the user.
• OTPs are commonly used in financial transactions or account changes to verify the identity of the user making the transaction. For example, when conducting a bank transfer or changing account details, an OTP might be sent to the registered mobile number or email address. The user enters the OTP to confirm the transaction, verifying that they are the account holder.
• OTPs are often used in password recovery processes to verify the user's identity. The service sends an OTP to the user's registered email address or mobile number, which the user then enters to verify their identity and proceed with resetting their password.
• When a user logs in from a new device, an OTP might be sent to their registered contact information. The user must enter the OTP to verify they have access to the registered device, confirming their identity and that the new device is trusted.

By integrating OTPs into authentication and verification processes, services can add an extra level of security and significantly reduce the risk of unauthorized access or identity theft.

Best practices for using one-time passwords

OTPs aren’t infallible, though. But the associated risks are generally around poor implementation rather than inherent to the method. To ensure OTP effectiveness and avoid some of the above potential vulnerabilities, you can follow some best practices.

Basic security practices

These practices are the principles of any good system:

• Secure delivery channel. Use a secure delivery channel for sending the OTP. If you're sending the OTP via email or SMS, ensure the communication channel is secure.
• Encrypt communication. Ensure all communications between the client and the server are done over HTTPS to prevent any interception of the OTP.
• Use secure storage. When storing OTPs on the server side, consider hashing them. This ensures that even if someone gains access to your storage, they cannot obtain the actual OTPs.

Good OTP design

These relate to how well you design your OTP algorithm and logic:

• Use a strong OTP. Use an OTP that is long and complex enough to resist brute-force attacks. An OTP with at least 6 digits is usually recommended.
• Time-Bound OTP. Make sure the OTP is valid only for a short period of time. This reduces the window an attacker has to use a stolen OTP. Usually, OTPs are valid for about 2-10 minutes.
• Limit OTP attempts. Implement rate limiting on your OTP endpoints to protect against brute force attacks. After a certain number of incorrect attempts, either block the user or implement a cool-down period.
• Expiry after use. The OTP should expire immediately after it has been used once, to prevent replay attacks.

Good UX

These help users use your OTP and make sure they don’t turn it off:

• Backup codes. For applications using OTP as a second factor, provide backup codes that the user can write down and use if they lose access to their OTP delivery method (like losing their phone).
• Fallback Options. In case the primary delivery channel fails (e.g., SMS not being delivered), have a secondary option like email or voice call.

Also consider educating users about using OTPs. One of the main threats to OTP use are physical–if an attacker steals a user's phone then they’ll have access to the SMS or email used with OTPs. Or a fraudster can fake a user’s identity to trick a telecoms company into assigning a new SIM with the user’s phone number to them (known as SIM swapping⁠).

While OTPs can greatly enhance security, they are not foolproof. Implementation within a broader security strategy is key.

Setting up OTP Authentication in Next.js

Let’s walk through setting up a one time password system within Next.js. If you already have a Next.js app up and running you can add this code directly. Otherwise create a new app using:

npx create-next-app@latest

We’ll also need to use a few modules to help us with our OTPs, namely:

• Twilio⁠. Twilio allows us to send SMS messages programmatically. Here we’re going to use it to send the OTP to the user's phone number via SMS.
• bcryptjs⁠. bcryptjs is a JavaScript library for hashing and comparing passwords. We’re going to use it to hash the OTP before storing it in the database for added security.
• MongoDB⁠. MongoDB is a NoSQL database that we’ll use to store hashed OTPs along with the associated phone numbers and expiry times.
• Upstash⁠. Upstash is a serverless data platform. We’re going to use it’s rate limiter and Redis functionality.

Install these with:

npm install twilio bcryptjs mongodb @upstash/ratelimit @upstash/redis

For Twilio and MongoDB, you’ll also need to sign up for accounts and then need your TWILIO_ACCOUNT_SID, your TWILIO_AUTH_TOKEN, and your MONGODB_URI. For Twilio, you’ll also need to buy a TWILIO_PHONE_NUMBER that will be used to send your SMS messages.

You’ll also need an account with Upstash, and then your UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN variables.

With that done, we’ll first create the API route that will generate our OTP:

// pages/api/generateOTP.js
import crypto from 'crypto'
import twilio from 'twilio'
import bcrypt from 'bcryptjs'
import { MongoClient } from 'mongodb'

export default async function handler(req, res) {
  if (req.method !== 'POST') {
    return res.status(405).end() // Method Not Allowed
  }

  // Generate a six digit number using the crypto module
  const otp = crypto.randomInt(100000, 999999)

  // Hash the OTP
  const hashedOtp = await bcrypt.hash(otp.toString(), 10)

  // Initialize the Twilio client
  const client = twilio(process.env.TWILIO_ACCOUNT_SID, process.env.TWILIO_AUTH_TOKEN)

  try {
    // Send the OTP via SMS
    await client.messages.create({
      body: `Your OTP is: ${otp}`,
      from: process.env.TWILIO_PHONE_NUMBER, // your Twilio number
      to: req.body.phone, // your user's phone number
    })

    // Store the hashed OTP in the database along with the phone number and expiry time
    const mongoClient = new MongoClient(process.env.MONGODB_URI)
    await mongoClient.connect()
    const otps = mongoClient.db().collection('otps')
    await otps.insertOne({
      phone: req.body.phone,
      otp: hashedOtp,
      expiry: Date.now() + 10 * 60 * 1000, // OTP expires after 10 minutes
    })
    await mongoClient.close()

    // Respond with a success status
    res.status(200).json({ success: true })
  } catch (err) {
    console.error(err)
    res.status(500).json({ error: 'Could not send OTP' })
  }
}

With this code we initially import all our dependencies, then create a handler function for our POST endpoint. The body of the POST request will contain the phone number of the user that we’ll get from the frontend. Within the endpoint, we’re doing a few things:

• Creating a six-digit random OTP
• Hashing that OTP with bcryptjs for storage
• Creating a Twilio client and then sending the OTP to the user’s phone number
• Creating a MongoDB client and storing the OTP along with the user’s phone number and an expiry time for the password.

Is this best practice? Absolutely not. We are doing a few things right, such as setting an expiry time on the OTP and hashing them. But our OTP generating ‘algorithm’ is laughably simple.

Let’s quickly create a frontend for this now:

import { useState } from 'react'

const OTPGenerator = () => {
  const [phone, setPhone] = useState('')
  const [otp, setOTP] = useState('')
  const [isLoading, setIsLoading] = useState(false)
  const [message, setMessage] = useState('')
  const [otpSent, setOtpSent] = useState(false)

  const handleSendOTP = async (event) => {
    event.preventDefault()
    setIsLoading(true)
    setMessage('') // reset message

    try {
      const response = await fetch('/api/generateOTP', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ phone }),
      })

      if (response.ok) {
        setMessage('OTP has been sent to your phone.')
        setOtpSent(true)
      } else {
        const data = await response.json()
        setMessage(data.error)
      }
    } catch (error) {
      setMessage('An error occurred. Please try again.')
      console.error(error)
    } finally {
      setIsLoading(false)
    }
  }

  const handleVerifyOTP = async (event) => {
    event.preventDefault()
    setIsLoading(true)
    setMessage('') // reset message

    try {
      const response = await fetch('/api/verifyOTP', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ phone, otp }),
      })

      if (response.ok) {
        setMessage('OTP verification successful!')
        setOtpSent(false)
        setPhone('')
        setOTP('')
      } else {
        const data = await response.json()
        setMessage(data.error)
      }
    } catch (error) {
      setMessage(error)
      console.error(error)
    } finally {
      setIsLoading(false)
    }
  }

  return (
    <div>
      {!otpSent ? (
        <form onSubmit={handleSendOTP}>
          <label>
            Phone Number:
            <input type="tel" value={phone} onChange={(e) => setPhone(e.target.value)} required />
          </label>
          <button type="submit" disabled={isLoading}>
            {isLoading ? 'Sending...' : 'Send OTP'}
          </button>
        </form>
      ) : (
        <form onSubmit={handleVerifyOTP}>
          <label>
            Enter OTP:
            <input type="text" value={otp} onChange={(e) => setOTP(e.target.value)} required />
          </label>
          <button type="submit" disabled={isLoading}>
            {isLoading ? 'Verifying...' : 'Verify OTP'}
          </button>
        </form>
      )}
      {message && <p>{message}</p>}
    </div>
  )
}

export default OTPGenerator

All this code will just show a single form on the page. On first load this form will ask for the user’s phone number.

When the user enters their phone number and hits submit, the above generateOTP endpoint will be called. This will send the OTP to the user’s phone number:

Hitting submit will also change the form to accept the OTP as the input. The user can then check their phone and enter the six-digit code: and hit submit again to send the OTP and phone number to a verifyOTP endpoint for verification:

// pages/api/verifyOTP.js
import bcrypt from 'bcryptjs'
import { MongoClient } from 'mongodb'
import { Ratelimit } from '@upstash/ratelimit'
import { Redis } from '@upstash/redis'

const rateLimiter = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.slidingWindow(2, '3 s'),
})

export default async function handler(req, res) {
  const user_ip = req.headers['x-forwarded-for']
  const { success } = await rateLimiter.limit(user_ip)

  if (!success) {
    return res.status(429).json({ error: 'Too Many Requests' })
  }

  if (req.method !== 'POST') {
    return res.status(405).end() // Method Not Allowed
  }

  const mongoClient = new MongoClient(process.env.MONGODB_URI)
  await mongoClient.connect()
  const otps = mongoClient.db().collection('otps')

  try {
    // Fetch the OTP record from the database
    const otpRecord = await otps.findOne({ phone: req.body.phone })

    if (!otpRecord) {
      return res.status(400).json({ error: 'Invalid phone number or OTP' })
    }

    // Check if the OTP has expired
    if (Date.now() > otpRecord.expiry) {
      return res.status(400).json({ error: 'OTP has expired' })
    }

    // Check if the OTPs match
    const otpMatch = await bcrypt.compare(req.body.otp.toString(), otpRecord.otp)
    if (!otpMatch) {
      return res.status(400).json({ error: 'Invalid phone number or OTP' })
    }

    // OTP is valid and has not expired, so we can delete it now
    await otps.deleteOne({ phone: req.body.phone })

    // Respond with a success status
    res.status(200).json({ success: true })
  } catch (err) {
    console.error(err)
    res.status(500).json({ error: 'Could not verify OTP' })
  } finally {
    await mongoClient.close()
  }
}

When called, this API loads the OTP MongoDB database and finds the one associated with the phone number. It checks whether it has expired, and if not, matches the hashed OTPs. If it's valid, the OTP gets deleted from the database and a 200 code is returned to the frontend.

We could then work that response into any other authentication flow we had set up. We also have a basic rate limiter set into that will make sure a user can’t input more than 2 codes in 3 seconds, to try and prevent brute force attacks:

And that’s it. You have one-time passwords working in Next.js.

OTPs are intricate but worth implementing

This code gives you a one-time password option for Next.js authentication. But we’ve only scratched the surface. We haven’t implemented this within any other authentication flow–this just generates, sends, and verifies the OTP, it doesn’t use it to authenticate the user.

We’ve also generated the OTP in the most basic manner, not following the RFCs and guidelines. We do have some nice best practices–rate-limiting, time-bounding, and expiry–but again these are all basic implementations. We also had to buy a new phone number!

This is the intricacy of OTPs. They are easy to set up, but difficult to get right. Like most authentication methods, it is better to use a provider than trying to create your own. Check out Clerk’s OTP solution here to have these intricacies taken care of for you.