Script command #2805

ChangeLog.md

@@ -6,6 +6,15 @@ Release notes:
 
 Major changes:
 
+* A new command, `script`, has been added, intended to make the script
+  interpreter workflow more reliable, easier to use, and more
+  efficient. This command forces the user to provide a `--resolver`
+  value, ignores all config files for more reproducible results, and
+  optimizes the existing package check to make the common case of all
+  packages already being present much faster. This mode does require
+  that all packages be present in a snapshot, however.
+  [#2805](https://github.com/commercialhaskell/stack/issues/2805)
+
 Behavior changes:
 
 * The default package metadata backend has been changed from Git to

doc/GUIDE.md

@@ -1646,23 +1646,17 @@ running the file.
 An example will be easiest to understand:
 
 ```
-michael@d30748af6d3d:~$ cat turtle.hs
+michael@d30748af6d3d:~$ cat turtle-example.hs
 #!/usr/bin/env stack
--- stack --resolver lts-3.2 --install-ghc runghc --package turtle
+-- stack --resolver lts-6.25 script --package turtle
 {-# LANGUAGE OverloadedStrings #-}
 import Turtle
 main = echo "Hello World!"
-michael@d30748af6d3d:~$ chmod +x turtle.hs
-michael@d30748af6d3d:~$ ./turtle.hs
-Run from outside a project, using implicit global project config
-Using resolver: lts-3.2 specified on command line
-hashable-1.2.3.3: configure
-# installs some more dependencies
-Completed all 22 actions.
+michael@d30748af6d3d:~$ chmod +x turtle-example.hs
+michael@d30748af6d3d:~$ ./turtle-example.hs
+Completed 5 action(s).
 Hello World!
-michael@d30748af6d3d:~$ ./turtle.hs
-Run from outside a project, using implicit global project config
-Using resolver: lts-3.2 specified on command line
+michael@d30748af6d3d:~$ ./turtle-example.hs
 Hello World!
 ```
 
@@ -1680,11 +1674,30 @@ ensure the turtle package is available.
 If you're on Windows: you can run `stack turtle.hs` instead of `./turtle.hs`.
 The shebang line is not required in that case.
 
+### Using multiple packages
+
+You can also specify multiple packages, either with multiple `--package`
+arguments, or by providing a comma or space separated list. For example:
+
+```
+#!/usr/bin/env stack
+{- stack
+  script
+  --resolver lts-6.25
+  --package turtle
+  --package "stm async"
+  --package http-client,http-conduit
+-}
+```
+
 ### Stack configuration for scripts
 
-If the current working directory is inside a project then that project's stack
-configuration is effective when running the script. Otherwise the script uses
-the global project configuration specified in
+With the `script` command, all Stack configuration files are ignored to provide a
+completely reliable script running experience. However, see the example below
+with `runghc` for an approach to scripts which will respect your configuration
+files. When using `runghc`, if the current working directory is inside a
+project then that project's stack configuration is effective when running the
+script. Otherwise the script uses the global project configuration specified in
 `~/.stack/global-project/stack.yaml`.
 
 ### Specifying interpreter options
@@ -1703,50 +1716,61 @@ separating the stack options and ghc options with a `--`. Here is an example of
 a multi line block comment with ghc options:
 
 ```
-  #!/usr/bin/env stack
-  {- stack
-    --resolver lts-3.2
-    --install-ghc
-    runghc
-    --package turtle
-    --
-    -hide-all-packages
-  -}
+#!/usr/bin/env stack
+{- stack
+  script
+  --resolver lts-6.25
+  --package turtle
+  --
+  +RTS -s -RTS
+-}
 ```
 
 ### Writing independent and reliable scripts
 
-Independent means that the script is independent of any prior deployment
-specific configuration. If required, the script will install everything it
-needs automatically on any machine that it runs on. To make a script always
-work irrespective of any specific environment configuration you can do the
-following:
+With the release of Stack 1.2.1, there is a new command, `script`, which will
+automatically:
+
+* Install GHC and libraries if missing
+* Require that all packages used be explicitly stated on the command line
+
+This ensures that your scripts are _independent_ of any prior deployment
+specific configuration, and are _reliable_ by using exactly the same version of
+all packages every time it runs so that the script does not break by
+accidentally using incompatible package versions.
+
+In previous versions of Stack, the `runghc` command was used for scripts
+instead. In order to achieve the same effect with the `runghc` command, you can
+do the following:
 
 1. Use the `--install-ghc` option to install the compiler automatically
 2. Explicitly specify all packages required by the script using the
 `--package` option. Use `-hide-all-packages` ghc option to force
 explicit specification of all packages.
+3. Use the `--resolver` Stack option to ensure a specific GHC version and
+   package set is used.
 
-Reliable means the script will use exactly the same version of all packages
-every time it runs so that the script does not break by accidentally using
-incompatible package versions. To achieve that use an explicit `--resolver`
-stack option.
-
-Here is an interpreter comment for a completely self-contained and reproducible
-version of our toy example:
-```
-  #!/usr/bin/env stack
-  {- stack
-    --resolver lts-3.2
-    --install-ghc
-    runghc
-    --package base
-    --package turtle
-    --
-    -hide-all-packages
+Even with this configuration, it is still possible for configuration
+files to impact `stack runghc`, which is why `stack script` is strongly
+recommended in general. For those curious, here is an example with `runghc`:
+
+```
+#!/usr/bin/env stack
+{- stack
+  --resolver lts-6.25
+  --install-ghc
+  runghc
+  --package base
+  --package turtle
+  --
+  -hide-all-packages
   -}
 ```
 
+The `runghc` command is still very useful, especially when you're working on a
+project and want to access the package databases and configurations used by
+that project. See the next section for more information on configuration files.
+
 ## Finding project configs, and the implicit global
 
 Whenever you run something with stack, it needs a `stack.yaml` project file. The

src/Stack/Build.hs

@@ -120,6 +120,11 @@ build setLocalFiles mbuildLk boptsCli = fixCodePage $ do
     plan <- withLoadPackage menv $ \loadPackage ->
         constructPlan mbp baseConfigOpts locals extraToBuild localDumpPkgs loadPackage sourceMap installedMap
 
+    allowLocals <- view $ configL.to configAllowLocals
+    unless allowLocals $ case justLocals plan of
+      [] -> return ()
+      localsIdents -> throwM $ LocalPackagesPresent localsIdents
+
     -- If our work to do is all local, let someone else have a turn with the snapshot.
     -- They won't damage what's already in there.
     case (mbuildLk, allLocal plan) of
@@ -155,6 +160,13 @@ allLocal =
     Map.elems .
     planTasks
 
+justLocals :: Plan -> [PackageIdentifier]
+justLocals =
+    map taskProvides .
+    filter ((== Local) . taskLocation) .
+    Map.elems .
+    planTasks
+
 checkCabalVersion :: (StackM env m, HasEnvConfig env) => m ()
 checkCabalVersion = do
     allowNewer <- view $ configL.to configAllowNewer

src/Stack/Build/ConstructPlan.hs

@@ -140,10 +140,7 @@ data Ctx = Ctx
 instance HasPlatform Ctx
 instance HasGHCVariant Ctx
 instance HasConfig Ctx
-instance HasBuildConfigNoLocal Ctx
 instance HasBuildConfig Ctx
-instance HasEnvConfigNoLocal Ctx where
-    envConfigNoLocalL = envConfigL.envConfigNoLocalL
 instance HasEnvConfig Ctx where
     envConfigL = lens ctxEnvConfig (\x y -> x { ctxEnvConfig = y })
 

src/Stack/Build/Source.hs

@@ -109,7 +109,7 @@ loadSourceMapFull omitWiredIn needTargets boptsCli = do
     -- Extend extra-deps to encompass targets requested on the command line
     -- that are not in the snapshot.
     extraDeps0 <- extendExtraDeps
-        (bcExtraDeps $ bcLocal bconfig)
+        (bcExtraDeps bconfig)
         cliExtraDeps
         (Map.keysSet $ Map.filter (== STUnknown) targets)
 
@@ -145,7 +145,7 @@ loadSourceMapFull omitWiredIn needTargets boptsCli = do
                 let flags =
                         case ( Map.lookup (Just n) $ boptsCLIFlags boptsCli
                              , Map.lookup Nothing $ boptsCLIFlags boptsCli
-                             , Map.lookup n $ unPackageFlags $ bcFlags $ bcLocal bconfig
+                             , Map.lookup n $ unPackageFlags $ bcFlags bconfig
                              ) of
                             -- Didn't have any flag overrides, fall back to the flags
                             -- defined in the snapshot.
@@ -195,7 +195,7 @@ getLocalFlags
 getLocalFlags bconfig boptsCli name = Map.unions
     [ Map.findWithDefault Map.empty (Just name) cliFlags
     , Map.findWithDefault Map.empty Nothing cliFlags
-    , Map.findWithDefault Map.empty name (unPackageFlags (bcFlags (bcLocal bconfig)))
+    , Map.findWithDefault Map.empty name (unPackageFlags (bcFlags bconfig))
     ]
   where
     cliFlags = boptsCLIFlags boptsCli
@@ -245,8 +245,7 @@ parseTargetsFromBuildOptsWith
     -> m (MiniBuildPlan, M.Map PackageName Version, M.Map PackageName SimpleTarget)
 parseTargetsFromBuildOptsWith rawLocals needTargets boptscli = do
     $logDebug "Parsing the targets"
-    bconfig <- view buildConfigNoLocalL
-    bconfigl <- view buildConfigLocalL
+    bconfig <- view buildConfigL
     mbp0 <-
         case bcResolver bconfig of
             ResolverCompiler _ -> do
@@ -264,16 +263,16 @@ parseTargetsFromBuildOptsWith rawLocals needTargets boptscli = do
     let snapshot = mpiVersion <$> mbpPackages mbp0
     flagExtraDeps <- convertSnapshotToExtra
         snapshot
-        (bcExtraDeps bconfigl)
+        (bcExtraDeps bconfig)
         rawLocals
         (catMaybes $ Map.keys $ boptsCLIFlags boptscli)
 
     (cliExtraDeps, targets) <-
         parseTargets
             needTargets
-            (bcImplicitGlobal bconfigl)
+            (bcImplicitGlobal bconfig)
             snapshot
-            (flagExtraDeps <> bcExtraDeps bconfigl)
+            (flagExtraDeps <> bcExtraDeps bconfig)
             (fst <$> rawLocals)
             workingDir
             (boptsCLITargets boptscli)
@@ -460,7 +459,7 @@ checkFlagsUsed :: (MonadThrow m, MonadReader env m, HasBuildConfig env)
                -> Map PackageName snapshot -- ^ snapshot, for error messages
                -> m ()
 checkFlagsUsed boptsCli lps extraDeps snapshot = do
-    bconfig <- view buildConfigLocalL
+    bconfig <- view buildConfigL
 
         -- Check if flags specified in stack.yaml and the command line are
         -- used, see https://github.com/commercialhaskell/stack/issues/617
@@ -512,7 +511,7 @@ extendExtraDeps extraDeps0 cliExtraDeps unknowns = do
     case errs of
         [] -> return $ Map.unions $ extraDeps1 : unknowns'
         _ -> do
-            bconfig <- view buildConfigLocalL
+            bconfig <- view buildConfigL
             throwM $ UnknownTargets
                 (Set.fromList errs)
                 Map.empty -- TODO check the cliExtraDeps for presence in index

src/Stack/BuildPlan.hs

@@ -32,6 +32,7 @@ module Stack.BuildPlan
     , showItems
     , showPackageFlags
     , parseCustomMiniBuildPlan
+    , loadBuildPlan
     ) where
 
 import           Control.Applicative
@@ -208,7 +209,7 @@ resolveBuildPlan
 resolveBuildPlan mbp isShadowed packages
     | Map.null (rsUnknown rs) && Map.null (rsShadowed rs) = return (rsToInstall rs, rsUsedBy rs)
     | otherwise = do
-        bconfig <- view buildConfigLocalL
+        bconfig <- view buildConfigL
         (caches, _gitShaCaches) <- getPackageCaches
         let maxVer =
                 Map.fromListWith max $