frigate/docs/docs/troubleshooting/memory.md

---
id: memory
title: Memory Troubleshooting
---

Frigate includes built-in memory profiling using [memray](https://bloomberg.github.io/memray/) to help diagnose memory issues. This feature allows you to profile specific Frigate modules to identify memory leaks, excessive allocations, or other memory-related problems.

## Enabling Memory Profiling

Memory profiling is controlled via the `FRIGATE_MEMRAY_MODULES` environment variable. Set it to a comma-separated list of module names you want to profile:

```bash
export FRIGATE_MEMRAY_MODULES="frigate.review_segment_manager,frigate.capture"
```

### Module Names

Frigate processes are named using a module-based naming scheme. Common module names include:

- `frigate.review_segment_manager` - Review segment processing
- `frigate.recording_manager` - Recording management
- `frigate.capture` - Camera capture processes (all cameras with this module name)
- `frigate.process` - Camera processing/tracking (all cameras with this module name)
- `frigate.output` - Output processing
- `frigate.audio_manager` - Audio processing
- `frigate.embeddings` - Embeddings processing

You can also specify the full process name (including camera-specific identifiers) if you want to profile a specific camera:

```bash
export FRIGATE_MEMRAY_MODULES="frigate.capture:front_door"
```

When you specify a module name (e.g., `frigate.capture`), all processes with that module prefix will be profiled. For example, `frigate.capture` will profile all camera capture processes.

## How It Works

1. **Binary File Creation**: When profiling is enabled, memray creates a binary file (`.bin`) in `/config/memray_reports/` that is updated continuously in real-time as the process runs.

2. **Automatic HTML Generation**: On normal process exit, Frigate automatically:

   - Stops memray tracking
   - Generates an HTML flamegraph report
   - Saves it to `/config/memray_reports/<module_name>.html`

3. **Crash Recovery**: If a process crashes (SIGKILL, segfault, etc.), the binary file is preserved with all data up to the crash point. You can manually generate the HTML report from the binary file.

## Viewing Reports

### Automatic Reports

After a process exits normally, you'll find HTML reports in `/config/memray_reports/`. Open these files in a web browser to view interactive flamegraphs showing memory usage patterns.

### Manual Report Generation

If a process crashes or you want to generate a report from an existing binary file, you can manually create the HTML report:

```bash
memray flamegraph /config/memray_reports/<module_name>.bin
```

This will generate an HTML file that you can open in your browser.

## Understanding the Reports

Memray flamegraphs show:

- **Memory allocations over time**: See where memory is being allocated in your code
- **Call stacks**: Understand the full call chain leading to allocations
- **Memory hotspots**: Identify functions or code paths that allocate the most memory
- **Memory leaks**: Spot patterns where memory is allocated but not freed

The interactive HTML reports allow you to:

- Zoom into specific time ranges
- Filter by function names
- View detailed allocation information
- Export data for further analysis

## Best Practices

1. **Profile During Issues**: Enable profiling when you're experiencing memory issues, not all the time, as it adds some overhead.

2. **Profile Specific Modules**: Instead of profiling everything, focus on the modules you suspect are causing issues.

3. **Let Processes Run**: Allow processes to run for a meaningful duration to capture representative memory usage patterns.

4. **Check Binary Files**: If HTML reports aren't generated automatically (e.g., after a crash), check for `.bin` files in `/config/memray_reports/` and generate reports manually.

5. **Compare Reports**: Generate reports at different times to compare memory usage patterns and identify trends.

## Troubleshooting

### No Reports Generated

- Check that the environment variable is set correctly
- Verify the module name matches exactly (case-sensitive)
- Check logs for memray-related errors
- Ensure `/config/memray_reports/` directory exists and is writable

### Process Crashed Before Report Generation

- Look for `.bin` files in `/config/memray_reports/`
- Manually generate HTML reports using: `memray flamegraph <file>.bin`
- The binary file contains all data up to the crash point

### Reports Show No Data

- Ensure the process ran long enough to generate meaningful data
- Check that memray is properly installed (included by default in Frigate)
- Verify the process actually started and ran (check process logs)

## Example Usage

```bash
# Enable profiling for review and capture modules
export FRIGATE_MEMRAY_MODULES="frigate.review_segment_manager,frigate.capture"

# Start Frigate
# ... let it run for a while ...

# Check for reports
ls -lh /config/memray_reports/

# If a process crashed, manually generate report
memray flamegraph /config/memray_reports/frigate_capture_front_door.bin
```

For more information about memray and interpreting reports, see the [official memray documentation](https://bloomberg.github.io/memray/).