Mastering Odoo 18: Our Epic Journey Through Real-time Chat & Cron Job Nightmares

A Battle Log of Debugging, Breakthroughs, and the Power of Persistence

As an IT company specializing in Odoo implementation and customization, Coqui Cloud often dives deep into the intricate world of Odoo deployments. Recently, we embarked on an epic troubleshooting journey with a client's Odoo 18 Community Edition setup, hosted on a robust Ubuntu server using Docker Compose, Caddy as a reverse proxy, and Cloudflare for DNS and CDN. What started as a seemingly straightforward real-time chat issue quickly unraveled into a complex web of interconnected problems, pushing our debugging skills to their limits.

This isn't just a technical guide; it's a battle log – an honest account of the frustrations, the missteps (even by AI assistants!), the critical discoveries, and the ultimate triumph of persistence. If you're running Odoo 18 in a multi-database Docker environment, this journey might save you countless hours.

The Initial Landscape: Odoo 18, Docker, Caddy, Cloudflare, and Two Core Problems

Our client's setup was designed for efficiency: a single primary Odoo instance Docker container serving multiple Odoo databases (one per domain, managed by dbfilter=^%h$) connected to a shared PostgreSQL 15 container (test18-db). Caddy handled all the reverse proxying and automatic SSL.

Despite the seemingly solid architecture, two major issues plagued the system:

1. Broken Real-time Chat (OdooBot/Discuss): Messages in Odoo Discuss (including OdooBot replies) would not appear instantly. A manual page refresh was always required to see new content. This pointed directly to a failure in Odoo's real-time communication (longpolling/WebSockets).
2. Persistent ir_cron Database Lock Errors: The PostgreSQL logs were frequently flooded with ERROR: could not obtain lock on row in relation "ir_cron". This indicated that Odoo's scheduled actions (cron jobs) were getting stuck, holding database locks, and causing general server instability and alarming memory spikes.

We knew these issues were interconnected, but finding the root causes required a systematic, and sometimes painful, process of elimination.

Phase 1: Battling the ir_cron Locks – A Deep Dive into Database Contention

The ir_cron errors were the most alarming, as they indicated core database instability.

Initial Symptoms & Misdirections:

• ERROR: could not obtain lock on row in relation "ir_cron": This was the primary message, often for "Mail: Fetchmail Service" (ID 6).
• Memory Spikes: htop showed RAM usage jumping from 2-3GB to over 11GB.
• Initial Suspects: We checked Odoo worker configurations (workers, max_cron_threads), which seemed reasonable. We also suspected external mail server issues.

Breakthrough 1: Cloudflare DNS for Mail – The First Crucial Correction

We discovered that several mail-related DNS records in Cloudflare (e.g., mail.domain.com, cpanel, webmail) were set to "Proxied" (orange cloud). This is a common misconfiguration for email protocols (IMAP/POP3/SMTP), as Cloudflare's proxy is for web traffic.

• Solution: Changed all mail-related A/CNAME records in Cloudflare to "DNS Only" (grey cloud). This allowed Odoo's Fetchmail to directly connect to mail servers.
• Result: This was a vital step for mail deliverability, but the ir_cron errors persisted.

Breakthrough 2: "Multiple Password Values Not Supported" – A Red Herring for the Root Cause

Ionblade, our hosting provider, reported this specific error in their mail server logs for admin@coqui.cloud and help@coqui.cloud. It meant Odoo was sending too many password values during authentication.

• Attempted Fix: Manually re-typed passwords in Odoo.
• Result: The specific "multiple password values" error might have cleared for some, but the ir_cron errors still persisted, even for mailboxes with only 7 emails and successful connection tests. This showed the problem wasn't just a simple password typo or mailbox size.

Breakthrough 3: RuntimeError: OrderedDict mutated during iteration – The True Root of Cron Instability

This was the most critical discovery for the ir_cron issue. This Python error appeared in Odoo's logs when cron jobs were active.

• What it means: Odoo's cron system was trying to loop through its list of active databases, but that list was unexpectedly changing or becoming unstable. This is a fundamental issue in Odoo's multi-database management, leading to cron job crashes and unreleased database locks.
• Initial Solution: Temporarily deactivated ALL scheduled actions (cron jobs) in Odoo across all instances.
• Result: The OrderedDict mutated errors and the ir_cron locking errors stopped completely. This confirmed the database was stable when cron jobs were inactive.

Breakthrough 4: WorkerCron Timeout – A Symptom of the OrderedDict Issue

Even after OrderedDict errors were gone, WorkerCron timeout after 2s errors appeared when Fetchmail was active. This was due to limit_time_real_cron = 2 in odoo.conf.

• Solution: Increased limit_time_real_cron to 120 seconds (2 minutes) in odoo.conf.
• Result: The WorkerCron timeout errors disappeared. This allowed cron jobs more time to execute, confirming the OrderedDict mutation was the underlying crash, not necessarily the cron job itself exceeding 2 seconds.

Current Status of ir_cron: The database is stable, and ir_cron errors are no longer appearing. This was a major victory!

Phase 2: The Stubborn Real-time Chat – Unraveling Odoo 18's Longpolling

With the database stable, we returned to the OdooBot/Discuss real-time chat issue.

Initial Diagnosis & Misdirections:

• Caddy Configuration: We extensively reviewed and adjusted Caddyfile settings for longpolling (handle /websocket, reverse_proxy, header_up Upgrade, header_up Connection). We even tried adding timeout directives (which crashed Caddy) and a websocket subdirective (which caused Caddy to fail).
• Odoo odoo.conf: Confirmed longpolling_port = 8072, proxy_mode = True.
• Network Checks: ss -tulnp | grep 18072 confirmed Docker was listening on the host port.
• Odoo Logs: log_level = debug showed no specific longpolling errors, only website.page logs.

Breakthrough 5: Direct IP Access Test – Ruling Out the Proxy Chain

A crucial test was accessing Odoo directly via its host IP and port (http://144.126.135.5:18069), bypassing Caddy and Cloudflare entirely.

• Result: The "Real-time connection lost..." message still appeared.
• Conclusion: This definitively ruled out Caddy, Cloudflare, and SSL as the cause of the real-time chat issue. The problem was internal to Odoo 18.

Breakthrough 6: Odoo 15 vs. Odoo 18 Longpolling – The Version Divide

A key observation from the client:

• Odoo 15 instances (e.g., coqui.cards, chinchorro.app) with similar Caddy configs worked perfectly for real-time chat.
• Odoo 18 instances (e.g., primary Odoo instance, demo18) did NOT work.

This strongly suggested the issue was specific to Odoo 18's longpolling implementation.

Breakthrough 7: ValueError: '{http.request.remote.ip}' in Odoo Logs – The Hidden Crash

While debugging the main test instance (a replica of primary Odoo instance), we found a critical error in Odoo's logs: ValueError: '{http.request.remote.ip}' does not appear to be an IPv4 or IPv6 address.

• What it means: Odoo's GeoIP module was receiving the literal string '{http.request.remote.ip}' instead of a valid IP address from Caddy. This was causing Odoo to crash internally (KeyError: 'ir.http'), which then broke longpolling.
• Solution: Corrected Caddy's header_up directives to use header_up X-Real-IP {http.request.header.Cf-Connecting-Ip} and header_up X-Forwarded-For {http.request.header.Cf-Connecting-Ip}. Cloudflare provides the real client IP in Cf-Connecting-Ip.
• Result: This fixed the ValueError and KeyError crashes, stabilizing Odoo's web server.

Breakthrough 8: The Comprehensive odoo.conf – The Final Piece for Real-time Chat

After fixing the IP header, the real-time chat still required a refresh. We then revisited a comprehensive odoo.conf from an Odoo forum post.

• Key Settings from Forum Post:
  • limit_memory_hard/soft, limit_request, limit_time_cpu, limit_time_real.
  • websocket_keep_alive_timeout = 600
  • websocket_rate_limit_burst = 10
  • websocket_rate_limit_delay = 0.2
  • workers = 3, max_cron_threads = 1 (adjusted from previous values).
• Result: After applying this odoo.conf (and reverting when it caused 502s due to too-low limits, then re-applying with careful checks), the real-time chat (OdooBot) started working! This was the final piece for the longpolling issue.

Key Takeaways & Lessons Learned from the Trenches

This journey taught us invaluable lessons about debugging complex Odoo deployments:

1. Systematic Isolation is Paramount: When multiple issues intertwine, isolate each component (Caddy, Odoo, Database, specific cron jobs, specific mailboxes) to pinpoint the root cause.
2. Logs are Your Best Friend (But Interpret Carefully): Detailed logs are essential, but understanding what each error truly means (e.g., WorkerCron timeout is a symptom, OrderedDict mutated is a cause) is critical.
3. Don't Assume (Even Your Own Knowledge): Always verify configurations, paths, and behaviors, even if they "should" be correct or "worked before." Assumptions lead to frustrating loops.
4. Odoo Version Nuances: Be aware that Odoo's internal workings (like longpolling/WebSockets) can change significantly between major versions (e.g., Odoo 15 vs. Odoo 18), requiring specific proxy or odoo.conf directives.
5. External Dependencies Matter: Mail servers, DNS (Cloudflare), and hosting provider firewalls can introduce subtle issues that manifest as Odoo-internal errors.
6. odoo.conf is Powerful (and Dangerous): Resource limits (limit_memory_hard, limit_time_real_cron), worker settings, and websocket_ parameters are crucial for stability and performance. Incorrect values can crash your instance.
7. Docker Volume Permissions: FileNotFoundError for sessions or filestore often points to incorrect permissions on Docker volumes, preventing Odoo from writing its data.
8. Community Wisdom is Gold: Forum posts (like the one for odoo.conf settings) can provide critical solutions, but they often need careful adaptation to your specific setup.

Conclusion: A Stable Odoo 18, Hard-Earned Expertise

After days of relentless troubleshooting, our client's Odoo 18 instance is now stable, performant, and fully functional:

• Real-time chat (OdooBot/Discuss) is working perfectly.
• Critical Odoo errors (FileNotFoundError, KeyError, OrderedDict mutated) are gone.
• ir_cron jobs are no longer timing out.
• Websites are accessible and responsive.

ODOO 18 CONFIGURATION FILE > odoo.conf < Using Caddy & Cloudflare

[options]
admin_passwd = Your_Very_Strong_Admin_P@$$u0rd
addons_path = /mnt/extra-addons,usr/lib/python3/dist-packages/odoo/addons
session_store = db
session_dir = False
limit_memory_hard = 2415919104
limit_memory_soft = 2013265920
limit_request = 8192
limit_time_cpu = 360
limit_real_time = 3600
limit_time_real_cron = 120
longpolling_port=8072
http_port =
max_cron_threads = 2
workers = 5
websocket_keep_alive_timeout = 600
websocket_rate_limit_burst = 10
websocket_rate_limit_delay = 0.2
xmlrpc = True
xmlrpc_interface =
xmlrpc_port = 8069
xmlrpcs = True
xmlrpcs_interface =
xmlrpcs_port = 8071
proxy_mode = True
dbfilter = ^%h$

Caddyfile

{
    email your@email.com

    # Enable logging to stdout so you can see it with `docker-compose logs caddy`
    log {
        output stdout
        level INFO
    }
}

# --- Odoo 18 Instances (Main Group) ---
coqui.cloud, www.coqui.cloud {
    encode zstd gzip

    reverse_proxy http://127.0.0.1:18069 {
        header_up X-Real-IP {http.request.header.Cf-Connecting-Ip}
        header_up X-Forwarded-For {http.request.header.Cf-Connecting-Ip}
        # Headers are automatically added by Caddy's reverse_proxy by default
    }
    # Odoo WebSocket / Longpolling traffic
    reverse_proxy /websocket {
        to http://127.0.0.1:18072
        header_up Host {host}
        header_up X-Real-IP {http.request.header.Cf-Connecting-Ip}
        header_up X-Forwarded-For {http.request.header.Cf-Connecting-Ip}
        header_up X-Forwarded-Host {host}
        header_up X-Forwarded-Proto {scheme}
        header_up Upgrade {http.request.header.Upgrade}
        header_up Connection {http.request.header.Connection}
    }
}

docker-compose.yml

services:
  odoo:
    image: ${ODOO_IMAGE}
    container_name: test18-odoo
    restart: unless-stopped
    ports:
      - "${ODOO_HOST_PORT}:8069"
      - "18072:8072" # <--- ADD THIS LINE
    depends_on:
      - db
    environment:
      - HOST=db
      - PORT=5432
      - USER=${POSTGRES_USER}
      - PASSWORD=${POSTGRES_PASSWORD}
      - TZ=${TZ}
      - ODOO_PROXY_MODE=True      
      - ODOO_DB_FILTER=^%h$

    volumes:
      # Mount Odoo configuration file (read-only)
      - ./config/odoo.conf:/etc/odoo/odoo.conf:ro
      # Mount named volume for Odoo data (filestore, sessions)
      - odoo_data:/var/lib/odoo
      # Mount SHARED addons directory from parent folder (read-only recommended)
      - ../addons:/mnt/extra-addons:ro
      # Mount host log directory for Odoo log file
      - ./logs:/var/log/odoo
      - ./config/GeoLite2-City.mmdb:/usr/share/GeoIP/GeoLite2-City.mmdb:ro
    networks:
      - odoo_net

  db:
    image: ${POSTGRES_IMAGE}
    container_name: test18-db
    restart: unless-stopped
    environment:
      - POSTGRES_USER=${POSTGRES_USER}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
      - POSTGRES_DB=${POSTGRES_DB}
      - PGDATA=/var/lib/postgresql/data/pgdata
      - TZ=${TZ}
    volumes:
      # Mount named volume for persistent PostgreSQL data
      - db_data:/var/lib/postgresql/data/pgdata
    networks:
      - odoo_net

volumes:
  odoo_data:
    name: ${ODOO_DATA_VOLUME_NAME} # e.g., test18-odoo-data-2204
  db_data:
    name: ${PGDATA_VOLUME_NAME} # e.g., test18-db-data-2204

networks:
  odoo_net:
    name: ${NETWORK_NAME} # e.g., test18-net-2204
    driver: bridge

.env

ODOO_IMAGE=coqui-odoo18:latest
ODOO_HOST_PORT=18001
POSTGRES_IMAGE=postgres:15
POSTGRES_DB=postgres
POSTGRES_USER=odoo_test_user
POSTGRES_PASSWORD=ChangMyTestPassword18!
PGDATA_VOLUME_NAME=test18-db-data
ODOO_DATA_VOLUME_NAME=test18-odoo-data
NETWORK_NAME=test18-net
TZ=America/Denver

This journey reinforced our commitment to meticulous debugging and the value of deep technical understanding. If you're facing similar challenges with your Odoo deployment, Coqui Cloud is here to help.