ESP32 FreeRTOS: tasks, queues and semaphores in plain English

Here’s something the Arduino documentation doesn’t tell you: when you call loop() on an ESP32, you’re running inside a FreeRTOS task. The ESP32’s Arduino core creates a task called loopTask and runs your setup() and loop() inside it. FreeRTOS is always running. You’re already using it. You’re just using it through a very thin abstraction that hides everything useful.

The moment your firmware needs to do two things with different timing — read a sensor every 100ms AND publish to MQTT whenever data is ready AND handle OTA without blocking either — you need to stop pretending the Arduino abstraction is sufficient and use FreeRTOS directly.

The lie of the Arduino loop

The Arduino programming model assumes your program does one thing at a time. In sequence. loop() executes, top to bottom, and starts again. This breaks down the moment you have any timing conflict.

Consider:

void loop() {
  // Read sensor every 100ms
  // Publish MQTT every 5000ms
  // Check for OTA updates
  // Handle serial commands
  // Blink a status LED at 2Hz
}

How do you read the sensor every exactly 100ms while also waiting for an MQTT publish that might take 300ms? You use millis() to track elapsed time and avoid delay(). You’ve now written a cooperative scheduler by hand. Except it’s not actually concurrent — if the MQTT publish blocks, the sensor read is late.

The ESP32 has two Xtensa LX6 cores. With FreeRTOS, you can run genuinely parallel tasks, each with their own stack, each scheduled preemptively. Here’s how.

Task creation with xTaskCreatePinnedToCore

BaseType_t result = xTaskCreatePinnedToCore(
  taskFunction,    // Function to run as the task
  "taskName",      // Descriptive name (for debugging only)
  stackDepth,      // Stack size in BYTES (not words, on ESP32)
  parameters,      // Void pointer passed to task function
  priority,        // 0 (lowest) to configMAX_PRIORITIES-1 (highest)
  &taskHandle,     // Out: handle for suspending/deleting/querying
  coreId           // 0 or 1; tskNO_AFFINITY for either core
);

if (result != pdPASS) {
  Serial.println("Task creation failed — out of heap?");
}

Task functions have a specific signature and must never return:

void myTask(void* parameter) {
  // Cast your parameter if you passed something
  SensorConfig* config = (SensorConfig*)parameter;

  // Initialization code for this task
  // ...

  while (true) {
    // Task body
    // ...

    // Always yield or delay — never spin-wait without giving up the CPU
    vTaskDelay(pdMS_TO_TICKS(100));  // Block for 100ms, let other tasks run
  }
  // Technically unreachable, but if you DO return:
  vTaskDelete(NULL);  // Delete self
}

If a task function returns without calling vTaskDelete(NULL), you get a hard fault. The FreeRTOS task infrastructure has no cleanup path for a returning task.

Stack size: the most common cause of mysterious crashes

Stack size is the parameter people guess at, get wrong, and then spend hours debugging obscure crashes. Here’s the mental model:

The stack holds: local variables, function call frames (return addresses, saved registers), and anything allocated with alloca(). Each nested function call consumes stack space. Deep call chains with large local arrays are the enemy.

What happens when stack overflows:

If stack overflow protection is enabled (CONFIG_FREERTOS_ENABLE_STATIC_TASK_CLEAN_UP or the default stack canary), FreeRTOS will call vApplicationStackOverflowHook() and the system prints an error and reboots. If protection is disabled, the stack silently corrupts adjacent memory — whatever is stored next to the stack in RAM gets overwritten. This produces the most baffling crashes: a variable in a completely unrelated task changes value for no reason, or a function pointer gets corrupted.

How to size stacks:

// At runtime, check how much stack a task has used
UBaseType_t highWaterMark = uxTaskGetStackHighWaterMark(taskHandle);
// Returns the MINIMUM free stack space since task start, in words (4 bytes each)
Serial.printf("Task high water mark: %u bytes free\n", highWaterMark * 4);

Start with 4096 bytes for tasks that don’t call complex library functions. If you’re calling WiFi, MQTT, JSON parsing, or any code with significant local variables, start at 8192. Run uxTaskGetStackHighWaterMark() in development and tune from there, leaving at least 512 bytes of headroom.

Task type	Suggested starting stack
Simple GPIO or timer task	2048 bytes
UART/serial processing	4096 bytes
WiFi/MQTT operations	8192 bytes
JSON serialization/deserialization	6144 bytes
HTTP client	10240 bytes

Task priorities

FreeRTOS priorities are numbers: 0 is idle (lowest), configMAX_PRIORITIES - 1 is highest (24 on default ESP32 config). The scheduler always runs the highest-priority ready task.

Warning: If a high-priority task never blocks (never calls vTaskDelay, xQueueReceive with a timeout, or similar), it starves all lower-priority tasks. The system appears to hang. The idle task (priority 0) will never run, and the watchdog (which monitors the idle task) will reset the chip.

Practical priority assignments:

Task	Priority
Idle task	0 (automatic)
Background MQTT publish	1
Sensor reading	2
WiFi event handling (system)	19
Time-critical control loop	10–15

The ESP32’s WiFi stack runs at high priority (around 19). Don’t run your tasks at 20+ unless you know exactly what you’re doing.

Why global variables for inter-task communication are wrong

The obvious way to share data between tasks:

// WRONG: unprotected shared state
float g_temperature = 0.0f;

void sensorTask(void* p) {
  while(true) {
    g_temperature = readSensor();  // Task A writes
    vTaskDelay(pdMS_TO_TICKS(100));
  }
}

void mqttTask(void* p) {
  while(true) {
    float t = g_temperature;  // Task B reads
    publishMQTT(t);
    vTaskDelay(pdMS_TO_TICKS(5000));
  }
}

On a dual-core processor, these tasks genuinely run simultaneously. On a 32-bit processor, reading or writing a float (which is 32 bits) is atomic at the hardware level — one instruction. So this particular case might actually be fine in practice.

But change that float to a struct:

struct SensorData {
  float temperature;
  float humidity;
  uint32_t timestamp;
};

SensorData g_data;  // 12 bytes — NOT atomically readable/writable

Now Task A is writing three fields. Task B can read after temperature is written but before humidity is written — it gets a mixed snapshot. This is a data race. It’s undefined behavior. It will produce subtle wrong values occasionally in production and be impossible to reproduce in development.

The correct solutions, in order of preference: queues for producer-consumer patterns, mutexes for shared resources that both tasks need to read and write.

Queues: the right way to pass data between tasks

A queue is a thread-safe FIFO buffer managed by FreeRTOS. The producer sends items into the queue; the consumer receives them. FreeRTOS handles all the synchronization.

// Create a queue that holds up to 10 SensorData items
QueueHandle_t sensorQueue = xQueueCreate(10, sizeof(SensorData));

void sensorTask(void* p) {
  while(true) {
    SensorData data;
    data.temperature = readTemperature();
    data.humidity    = readHumidity();
    data.timestamp   = millis();

    // Send to queue; don't wait if queue is full (pdMS_TO_TICKS(0) = no wait)
    if (xQueueSend(sensorQueue, &data, pdMS_TO_TICKS(0)) != pdTRUE) {
      Serial.println("[Sensor] Queue full — dropping reading");
    }

    vTaskDelay(pdMS_TO_TICKS(100));
  }
}

void mqttTask(void* p) {
  SensorData data;
  while(true) {
    // Block until a reading is available (up to 10 seconds)
    if (xQueueReceive(sensorQueue, &data, pdMS_TO_TICKS(10000)) == pdTRUE) {
      publishSensorData(data);
    }
    // If timeout, loop again — connection check happens inside publishSensorData
  }
}

The queue length (10 in this example) provides buffering for timing jitter. If the MQTT publish takes 2 seconds due to a slow broker, the sensor task has already put 20 more readings in the queue. The consumer will catch up.

Mutexes: protecting shared resources

A mutex (mutual exclusion lock) allows only one task at a time to execute a critical section. Use it when multiple tasks need to read and write shared state, and you can’t replace the shared state with a queue.

SemaphoreHandle_t displayMutex = xSemaphoreCreateMutex();

void updateDisplay(const char* line1, const char* line2) {
  // Take the mutex; wait up to 100ms
  if (xSemaphoreTake(displayMutex, pdMS_TO_TICKS(100)) == pdTRUE) {
    lcd.setCursor(0, 0);
    lcd.print(line1);
    lcd.setCursor(0, 1);
    lcd.print(line2);
    xSemaphoreGive(displayMutex);  // Always release — or everything deadlocks
  } else {
    Serial.println("[Display] Mutex timeout — display busy");
  }
}

The critical mistake: taking a mutex and then calling something that blocks without giving the mutex back first. Any code path that might return early (error handling, early return) must release the mutex.

Semaphores: signaling between tasks

A counting semaphore is a signaling mechanism. One task waits on the semaphore; another task “gives” it to wake the waiter.

SemaphoreHandle_t dataReadySemaphore = xSemaphoreCreateBinary();

void IRAM_ATTR dataReadyISR() {
  BaseType_t higherPriorityTaskWoken = pdFALSE;
  xSemaphoreGiveFromISR(dataReadySemaphore, &higherPriorityTaskWoken);
  portYIELD_FROM_ISR(higherPriorityTaskWoken);  // Request context switch if needed
}

void processingTask(void* p) {
  while(true) {
    // Block forever waiting for the semaphore to be given
    if (xSemaphoreTake(dataReadySemaphore, portMAX_DELAY) == pdTRUE) {
      processNewData();
    }
  }
}

The portYIELD_FROM_ISR() call at the end of the ISR tells the FreeRTOS scheduler to consider switching to the newly-unblocked task immediately rather than waiting for the next tick. For low-latency response, this matters.

Common mistakes

Stack overflow: Too small a stack. Use uxTaskGetStackHighWaterMark() to find out.

Blocking in an ISR: Calling xQueueSend() (without FromISR) from an interrupt. Use the FromISR variants always.

Priority inversion: Task A (low priority) holds a mutex. Task B (high priority) waits for it. Task C (medium priority) runs forever because it preempts Task A but doesn’t need the mutex. Task A never runs, never releases the mutex, Task B waits forever. Fix: use xSemaphoreCreateMutex() (which creates a priority-inheriting mutex) not a binary semaphore for shared resource protection.

Forgetting to call vTaskDelete(NULL): Task functions that return (should never happen, but defensive programming) must call this.

Complete example: dual-core sensor-to-MQTT pipeline

#include <Arduino.h>
#include <WiFi.h>
#include <PubSubClient.h>
#include <ArduinoJson.h>

struct SensorReading {
  float temperature;
  float humidity;
  uint32_t timestamp_ms;
};

QueueHandle_t readingQueue;
WiFiClient   wifiClient;
PubSubClient mqtt(wifiClient);

// --- Core 0: Sensor reading task ---

void sensorTask(void* param) {
  Serial.printf("[Sensor] Task started on core %d\n", xPortGetCoreID());

  while (true) {
    SensorReading r;
    r.temperature  = 22.5f + (float)(random(-20, 20)) / 10.0f;  // Simulated
    r.humidity     = 55.0f + (float)(random(-50, 50)) / 10.0f;
    r.timestamp_ms = millis();

    if (xQueueSend(readingQueue, &r, pdMS_TO_TICKS(50)) != pdTRUE) {
      Serial.println("[Sensor] Queue full, dropping reading");
    }

    // Log stack usage every 60 seconds in development
    static uint32_t lastStackCheck = 0;
    if (millis() - lastStackCheck > 60000) {
      lastStackCheck = millis();
      Serial.printf("[Sensor] Stack HWM: %u bytes\n",
                    uxTaskGetStackHighWaterMark(NULL) * 4);
    }

    vTaskDelay(pdMS_TO_TICKS(100));  // Read every 100ms
  }
}

// --- Core 1: MQTT task ---

bool ensureConnected() {
  if (WiFi.status() != WL_CONNECTED) {
    return false;
  }
  if (!mqtt.connected()) {
    mqtt.connect("esp32-demo");
    return mqtt.connected();
  }
  return true;
}

void mqttTask(void* param) {
  Serial.printf("[MQTT] Task started on core %d\n", xPortGetCoreID());
  mqtt.setServer("192.168.1.100", 1883);
  mqtt.setBufferSize(1024);

  SensorReading r;
  while (true) {
    mqtt.loop();

    // Drain the queue, publish each reading
    while (xQueueReceive(readingQueue, &r, 0) == pdTRUE) {
      if (!ensureConnected()) {
        // If can't connect, discard this reading and move on
        Serial.println("[MQTT] Not connected, discarding reading");
        continue;
      }

      StaticJsonDocument<256> doc;
      doc["temp"]      = r.temperature;
      doc["humi"]      = r.humidity;
      doc["uptime_ms"] = r.timestamp_ms;

      char payload[256];
      serializeJson(doc, payload, sizeof(payload));
      mqtt.publish("sensors/demo/telemetry", payload);
    }

    vTaskDelay(pdMS_TO_TICKS(10));
  }
}

void setup() {
  Serial.begin(115200);

  WiFi.begin("your-ssid", "your-password");
  while (WiFi.status() != WL_CONNECTED) {
    vTaskDelay(pdMS_TO_TICKS(500));
    Serial.print(".");
  }
  Serial.println("\n[WiFi] Connected");

  // Queue holds 20 readings — enough buffer for a multi-second MQTT hiccup
  readingQueue = xQueueCreate(20, sizeof(SensorReading));
  configASSERT(readingQueue != NULL);

  // Sensor task on Core 0, priority 2, 4KB stack
  xTaskCreatePinnedToCore(sensorTask, "SensorTask", 4096, NULL, 2, NULL, 0);

  // MQTT task on Core 1, priority 1, 8KB stack (MQTT needs more stack)
  xTaskCreatePinnedToCore(mqttTask, "MQTTTask", 8192, NULL, 1, NULL, 1);
}

void loop() {
  // Arduino's loopTask still runs — yield to avoid starving the system
  vTaskDelay(pdMS_TO_TICKS(1000));
}

Run this and you’ll see the sensor task firing every 100ms on core 0, while the MQTT task on core 1 drains the queue and publishes. If MQTT is slow, readings buffer in the queue. If the queue fills up (MQTT is down for more than 2 seconds at 100ms/reading), readings are dropped with a log message — which is the correct behavior for telemetry data.

What’s next

If you’re deploying this device to a location you can’t physically reach, you need to solve firmware updates. The ESP32 OTA updates post covers the ESP32 partition scheme, rolling back from failed updates, and securing the OTA endpoint — the pieces that turn OTA from a development convenience into a production-grade deployment mechanism.