Elixir Wizards

"You've Got a Job to Do" with Joel Meador


Listen Later

Today on Elixir Wizards Office Hours, SmartLogic Engineer Joel Meador joins Dan Ivovich to discuss all things background jobs.

The behind-the-scenes heroes of app performance and scalability, background jobs take center stage as we dissect their role in optimizing user experience and managing heavy-lifting tasks away from the main application flow. From syncing with external systems to processing large datasets, background jobs are pivotal to successful application management.

Dan and Joel share their perspectives on monitoring, debugging, and securing background jobs, emphasizing the need for a strategic approach to these hidden workflows.

Key topics discussed in this episode:
  • The vital role of background jobs in app performance
  • Optimizing user experience through background processing
  • Common pitfalls: resource starvation and latency issues
  • Strategies for effective monitoring and debugging of task runners and job schedulers
  • Data integrity and system security in open source software
  • Background job tools like Oban, Sidekiq, Resque, Cron jobs, Redis pub sub
  • CPU utilization and processing speed
  • Best practices for implementing background jobs
  • Keeping jobs small, focused, and well-monitored
  • Navigating job uniqueness, locking, and deployment orchestration
  • Leveraging asynctask for asynchronous operations
  • The art of continuous improvement in background job management
  • Links mentioned in this episode:

    https://redis.io/

    Oban job processing library https://hexdocs.pm/oban/Oban.html
    Resque Ruby library for background jobs https://github.com/resque
    Sidekiq background processing for Ruby https://github.com/sidekiq
    Delayed Job priority queue system https://github.com/collectiveidea/delayed_job
    RabbitMQ messaging and streaming broker https://www.rabbitmq.com/
    Mnesia distributed telecommunications DBMS https://www.erlang.org/doc/man/mnesia.html
    Task for Elixir https://hexdocs.pm/elixir/1.12/Task.html
    ETS in-memory store for Elixir and Erlang objects https://hexdocs.pm/ets/ETS.html
    Cron - https://en.wikipedia.org/wiki/Cron
    Donate to Miami Indians of Indiana https://www.miamiindians.org/take-action
    Joel Meador on Tumblr https://joelmeador.tumblr.com/

    Special Guest: Joel Meador.

    ...more
    View all episodesView all episodes
    Download on the App Store

    Elixir WizardsBy SmartLogic LLC

    • 4.9
    • 4.9
    • 4.9
    • 4.9
    • 4.9

    4.9

    22 ratings


    More shows like Elixir Wizards

    View all
    Software Engineering Radio - the podcast for professional software developers by se-radio@computer.org

    Software Engineering Radio - the podcast for professional software developers

    262 Listeners

    The Changelog: Software Development, Open Source by Changelog Media

    The Changelog: Software Development, Open Source

    286 Listeners

    Startups For the Rest of Us by Rob Walling

    Startups For the Rest of Us

    694 Listeners

    Software Engineering Daily by Software Engineering Daily

    Software Engineering Daily

    630 Listeners

    Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

    Syntax - Tasty Web Development Treats

    987 Listeners

    REWORK by 37signals

    REWORK

    211 Listeners

    CoRecursive: Coding Stories by Adam Gordon Bell - Software Developer

    CoRecursive: Coding Stories

    185 Listeners

    Elixir Mix by Charles M Wood

    Elixir Mix

    13 Listeners

    The Stack Overflow Podcast by The Stack Overflow Podcast

    The Stack Overflow Podcast

    63 Listeners

    Thinking Elixir Podcast by ThinkingElixir.com

    Thinking Elixir Podcast

    31 Listeners

    Beam Radio by Lars Wikman

    Beam Radio

    11 Listeners

    Software Unscripted by Richard Feldman

    Software Unscripted

    26 Listeners

    Oxide and Friends by Oxide Computer Company

    Oxide and Friends

    47 Listeners

    Elixir Mentor by Jacob Luetzow

    Elixir Mentor

    2 Listeners

    Elixir Friends by German Velasco

    Elixir Friends

    0 Listeners