Skip to content

Commit

Permalink
Symlink packages after composer update
Browse files Browse the repository at this point in the history
This is a fundamental rewrite of the Composer integration. Now, instead
of adding the loaded paths to Composer's search path (by creating path
repositories for them), we replace the packages downloaded by Composer
that can be found in the loaded paths by symlinks to the local paths.

Doing so requires us to hook into the autoload dumper, which now has to
respect the rules in the local path, not those obtained from Packagist.

All of this should hopefully fix several issues, most importantly:
- Composer's lock file will be written before Studio does its magic,
  therefore not causing any conflicts with other developers' setups.
- Different version constraints on symlinked packages won't cause
  problems anymore. Any required packages that are found in loaded paths
  will be loaded, no matter the branch or version they are on.

Open questions:
- How should packages be handled that have not yet been added to
  Packagist? (Proposed solution: Create path repositories for the loaded
  paths, but *append* them instead of *prepending*, so that they will
  only be used as fallback, if Packagist does not yield any results.)
- Should we validate the constraints from composer.json before creating
  symlinks? With this setup, everything might be working locally, but
  not when downloading the package from Packagist (as another version
  may be downloaded instead).

Refs #52, #58, #65, #72.
  • Loading branch information
franzliedke committed Dec 2, 2017
1 parent 4c8784d commit d4e19eb
Showing 1 changed file with 111 additions and 12 deletions.
123 changes: 111 additions & 12 deletions src/Composer/StudioPlugin.php
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,16 @@
use Composer\Composer;
use Composer\EventDispatcher\EventSubscriberInterface;
use Composer\IO\IOInterface;
use Composer\Json\JsonFile;
use Composer\Package\PackageInterface;
use Composer\Plugin\PluginInterface;
use Composer\Repository\CompositeRepository;
use Composer\Repository\InstalledFilesystemRepository;
use Composer\Repository\PathRepository;
use Composer\Repository\WritableRepositoryInterface;
use Composer\Script\ScriptEvents;
use Composer\Util\Filesystem;
use Studio\Config\Config;
use Studio\Config\FileStorage;

class StudioPlugin implements PluginInterface, EventSubscriberInterface
{
Expand All @@ -31,32 +36,121 @@ public function activate(Composer $composer, IOInterface $io)

public static function getSubscribedEvents()
{
// TODO: Before update, append Studio path repositories
return [
ScriptEvents::PRE_INSTALL_CMD => 'registerStudioPackages',
ScriptEvents::PRE_UPDATE_CMD => 'registerStudioPackages',
ScriptEvents::POST_UPDATE_CMD => 'symlinkStudioPackages',
ScriptEvents::PRE_AUTOLOAD_DUMP => 'loadStudioPackagesForDump',
];
}

/**
* Register all managed paths with Composer.
* Symlink all Studio-managed packages
*
* This function configures Composer to treat all Studio-managed paths as local path repositories, so that packages
* therein will be symlinked directly.
* After `composer update`, we replace all packages that can also be found
* in paths managed by Studio with symlinks to those paths.
*/
public function registerStudioPackages()
public function symlinkStudioPackages()
{
$intersection = $this->getManagedPackages();

// Create symlinks for all left-over packages in vendor/composer/studio
$destination = $this->composer->getConfig()->get('vendor-dir') . '/composer/studio';
(new Filesystem())->emptyDirectory($destination);
$studioRepo = new InstalledFilesystemRepository(
new JsonFile($destination . '/installed.json')
);

$installationManager = $this->composer->getInstallationManager();

// Get local repository which contains all installed packages
$installed = $this->composer->getRepositoryManager()->getLocalRepository();

foreach ($intersection as $package) {
$original = $installed->findPackage($package->getName(), '*');

$installationManager->getInstaller($original->getType())
->uninstall($installed, $original);

$installationManager->getInstaller($package->getType())
->install($studioRepo, $package);
}

$studioRepo->write();

// TODO: Run dump-autoload again
}

public function loadStudioPackagesForDump()
{
$localRepo = $this->composer->getRepositoryManager()->getLocalRepository();
$intersection = $this->getManagedPackages();

$packagesToReplace = [];
foreach ($intersection as $package) {
$packagesToReplace[] = $package->getName();
}

// Remove all packages with same names as one of symlinked packages
$packagesToRemove = [];
foreach ($localRepo->getCanonicalPackages() as $package) {
if (in_array($package->getName(), $packagesToReplace)) {
$packagesToRemove[] = $package;
}
}
foreach ($packagesToRemove as $package) {
$localRepo->removePackage($package);
}

// Add symlinked packages to local repository
foreach ($intersection as $package) {
$localRepo->addPackage(clone $package);
}
}

/**
* @param WritableRepositoryInterface $installedRepo
* @param PathRepository[] $managedRepos
* @return PackageInterface[]
*/
private function getIntersection(WritableRepositoryInterface $installedRepo, $managedRepos)
{
$managedRepo = new CompositeRepository($managedRepos);

return array_filter(
array_map(
function (PackageInterface $package) use ($managedRepo) {
return $managedRepo->findPackage($package->getName(), '*');
},
$installedRepo->getCanonicalPackages()
)
);
}

private function getManagedPackages()
{
$repoManager = $this->composer->getRepositoryManager();
$composerConfig = $this->composer->getConfig();

// Get array of PathRepository instances for Studio-managed paths
$managed = [];
foreach ($this->getManagedPaths() as $path) {
$this->io->writeError("[Studio] Loading path $path");

$repoManager->prependRepository(new PathRepository(
$managed[] = new PathRepository(
['url' => $path],
$this->io,
$composerConfig
));
);
}

// Intersect PathRepository packages with local repository
$intersection = $this->getIntersection(
$this->composer->getRepositoryManager()->getLocalRepository(),
$managed
);

foreach ($intersection as $package) {
$this->write('Loading package ' . $package->getUniqueName());
}

return $intersection;
}

/**
Expand All @@ -71,4 +165,9 @@ private function getManagedPaths()

return $config->getPaths();
}

private function write($msg)
{
$this->io->writeError("[Studio] $msg");
}
}

5 comments on commit d4e19eb

@ElfSundae
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can not work with wildcard path.

@ElfSundae
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[InvalidArgumentException]
Package foo/bar-9999999-dev seems not been installed properly

@ElfSundae
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

errored when composer update, but succeed if I run composer update --no-autoloader && composer dump-autoload

Maybe symlinkStudioPackages and loadStudioPackagesForDump are not concerted?

@julienfalque
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for me: the plugin isn't able to symlink the package I want to use. It fails whether required version in the main composer.json matches the git branch of the package I want to symlink or not, even when overriding it in the package's composer.json file with "version": "dev-some-branch-name" as I had to do with version 0.12 of the plugin.

Using the --no-autoloader option makes the plugin work as expected in all these cases.

@ElfSundae
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@franzliedke
Did you try the Installer Events? I am not familiar with the Composer plugin development, after some black-box logging tests, I guess the following may be work:

  • post-dependencies-solving: symlink Studio-managed packages
  • pre-dependencies-solving: restore Studio-managed paths (symlinks) to the origin packages, to let Composer install/update work correctly

I think we just need to replace (symlink) Studio-managed packages during dependencies solving, then the Composer process can be:

  1. Solve dependencies, download/install packages
  2. Generate/Update composer.lock file
  3. Symlink Studio-managed packages to local paths
  4. Dump autoloader file as normal
  5. Run post-package-* events handers if any

Please sign in to comment.