Package: crew 1.0.0

William Michael Landau

crew: A Distributed Worker Launcher Framework

In computationally demanding analysis projects, statisticians and data scientists asynchronously deploy long-running tasks to distributed systems, ranging from traditional clusters to cloud services. The 'NNG'-powered 'mirai' R package by Gao (2023) <doi:10.5281/zenodo.7912722> is a sleek and sophisticated scheduler that efficiently processes these intense workloads. The 'crew' package extends 'mirai' with a unifying interface for third-party worker launchers. Inspiration also comes from packages. 'future' by Bengtsson (2021) <doi:10.32614/RJ-2021-048>, 'rrq' by FitzJohn and Ashton (2023) <https://github.com/mrc-ide/rrq>, 'clustermq' by Schubert (2019) <doi:10.1093/bioinformatics/btz284>), and 'batchtools' by Lang, Bischel, and Surmann (2017) <doi:10.21105/joss.00135>.

Authors:William Michael Landau [aut, cre], Daniel Woodie [ctb], Eli Lilly and Company [cph, fnd]

crew_1.0.0.tar.gz
crew_1.0.0.zip(r-4.5)crew_1.0.0.zip(r-4.4)crew_1.0.0.zip(r-4.3)
crew_1.0.0.tgz(r-4.5-any)crew_1.0.0.tgz(r-4.4-any)crew_1.0.0.tgz(r-4.3-any)
crew_1.0.0.tar.gz(r-4.5-noble)crew_1.0.0.tar.gz(r-4.4-noble)
crew_1.0.0.tgz(r-4.4-emscripten)crew_1.0.0.tgz(r-4.3-emscripten)
crew.pdf |crew.html✨
crew/json (API)
NEWS

# Install 'crew' in R:
install.packages('crew', repos = c('https://test.r-universe.dev', 'https://cloud.r-project.org'))

Bug tracker:https://github.com/wlandau/crew/issues

Pkgdown site:https://wlandau.github.io

On CRAN:

high-performance-computing

11.11 score 133 stars 2 packages 221 scripts 2.0k downloads 45 exports 24 dependencies

Last updated 21 days agofrom:e28d0c782f (on 1.0.0). Checks:8 OK. Indexed: no.

TargetResultLatest binary
Doc / VignettesOKFeb 03 2025
R-4.5-winOKFeb 03 2025
R-4.5-macOKFeb 03 2025
R-4.5-linuxOKFeb 03 2025
R-4.4-winOKFeb 03 2025
R-4.4-macOKFeb 03 2025
R-4.3-winOKFeb 03 2025
R-4.3-macOKFeb 03 2025

Exports:all_ofany_ofcontainscrew_assertcrew_asynccrew_class_asynccrew_class_clientcrew_class_controllercrew_class_controller_groupcrew_class_launchercrew_class_launcher_localcrew_class_monitor_localcrew_class_queuecrew_class_relaycrew_class_throttlecrew_class_tlscrew_cleancrew_clientcrew_controllercrew_controller_groupcrew_controller_localcrew_deprecatecrew_evalcrew_eval_asynccrew_launchercrew_launcher_localcrew_monitor_localcrew_options_localcrew_options_metricscrew_queuecrew_random_namecrew_relaycrew_retrycrew_terminate_processcrew_terminate_signalcrew_throttlecrew_tlscrew_workerends_witheverythinglast_colmatchesnum_rangeone_ofstarts_with

Dependencies:clidata.tablefansifastmapgetipgluelaterlifecyclemagrittrmirainanonextpillarpkgconfigprocessxpromisespsR6Rcpprlangtibbletidyselectutf8vctrswithr

Asynchronous Shiny apps

Will Landau and Daniel Woodie

Rendered fromshiny.Rmdusingknitr::rmarkdownon Feb 03 2025.

Last update: 2024-08-01
Started: 2023-05-02

Controller groups

Rendered fromgroups.Rmdusingknitr::rmarkdownon Feb 03 2025.

Last update: 2025-01-28
Started: 2023-06-16

Integration with promises

Rendered frompromises.Rmdusingknitr::rmarkdownon Feb 03 2025.

Last update: 2024-05-03
Started: 2024-02-06

Introduction to crew

Rendered fromintroduction.Rmdusingknitr::rmarkdownon Feb 03 2025.

Last update: 2025-01-28
Started: 2023-09-14

Known risks of crew

Rendered fromrisks.Rmdusingknitr::rmarkdownon Feb 03 2025.

Last update: 2024-10-08
Started: 2023-09-14

Launcher plugins

Rendered fromplugins.Rmdusingknitr::rmarkdownon Feb 03 2025.

Last update: 2025-01-28
Started: 2023-03-21

Logging

Rendered fromlogging.Rmdusingknitr::rmarkdownon Feb 03 2025.

Last update: 2024-10-22
Started: 2024-10-08

Readme and manuals

Help Manual

Help pageTopics
crew: a distributed worker launcher frameworkcrew-package
Crew assertioncrew_assert
Local asynchronous client object.crew_async
'R6' async class.crew_class_async
'R6' client class.crew_class_client
Controller classcrew_class_controller
Controller group classcrew_class_controller_group
Launcher abstract classcrew_class_launcher
Local process launcher classcrew_class_launcher_local
Local monitor classcrew_class_monitor_local
'R6' queue classcrew_class_queue
'R6' relay class.crew_class_relay
'R6' throttle class.crew_class_throttle
'R6' TLS class.crew_class_tls
Terminate dispatchers and/or workerscrew_clean
Create a client object.crew_client
Create a controller object from a client and launcher.crew_controller
Create a controller group.crew_controller_group
Create a controller with a local process launcher.crew_controller_local
Deprecate a 'crew' feature.crew_deprecate
Evaluate an R command and return results as a monad.crew_eval
Create an abstract launcher.crew_launcher
Create a launcher with local process workers.crew_launcher_local
Create a local monitor object.crew_monitor_local
Local 'crew' launcher options.crew_options_local
Options for logging resource usage metrics.crew_options_metrics
Create a 'crew' queue object.crew_queue
Random namecrew_random_name
Create a 'crew' relay object.crew_relay
Retry code.crew_retry
Manually terminate a local process.crew_terminate_process
Get the termination signal.crew_terminate_signal
Create a stateful throttling object.crew_throttle
Configure TLS.crew_tls
Crew worker.crew_worker