Practically Correct, Just-in-Time Shell Script Parallelization

Konstantinos Kallas, Tammam Mustafa, Jan Bielak, Dimitris Karnikis, Thurston H.Y. Dang, Michael Greenberg, Nikos Vasilakis

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations

Abstract

Recent shell-script parallelization systems enjoy mostly automated speedups by parallelizing scripts ahead-of-time. Unfortunately, such static parallelization is hampered by dynamic behavior pervasive in shell scripts-e.g., variable expansion and command substitution-which often requires reasoning about the current state of the shell and filesystem. We present a just-in-time (JIT) shell-script compiler, PASH-JIT, that intermixes evaluation and parallelization during a script's run-time execution. JIT parallelization collects runtime information about the system's state, but must not alter the behavior of the original script and must maintain minimal overhead. PASH-JIT addresses these challenges by (1) using a dynamic interposition framework, guided by a static preprocessing pass, (2) developing runtime support for transparently pausing and resuming shell execution; and (3) operating as a stateful server, communicating with the current shell by passing messages-all without requiring modifications to the system's underlying shell interpreter. When run on a wide variety of benchmarks, including the POSIX shell test suite, PASH-JIT (1) does not break scripts, even in cases that are likely to break shells in widespread use; and (2) offers significant speedups, whenever parallelization is possible. These results show that PASH-JIT can be used as a drop-in replacement for any non-interactive shell use, providing significant speedups without any risk of breakage.

Original languageEnglish
Title of host publicationProceedings of the 16th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2022
Pages769-785
Number of pages17
ISBN (Electronic)9781939133281
StatePublished - 2022
Event16th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2022 - Carlsbad, United States
Duration: 11 Jul 202213 Jul 2022

Publication series

NameProceedings of the 16th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2022

Conference

Conference16th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2022
Country/TerritoryUnited States
CityCarlsbad
Period11/07/2213/07/22

Fingerprint

Dive into the research topics of 'Practically Correct, Just-in-Time Shell Script Parallelization'. Together they form a unique fingerprint.

Cite this