mdebray/tvl-depot
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Support mandatory system features in the build hook

Browse source 
Mandatory features are features that MUST be present in a derivation's
requiredSystemFeatures attribute.  One application is performance
testing, where we have a dedicated machine to run performance tests
(and nothing else).  Then we would add the label "perf" to the
machine's mandatory features and to the performance testing
derivations.
This commit is contained in:
Eelco Dolstra 2012-04-30 17:22:45 -04:00
parent 82ae0e688c
commit 59a26360c7
2 changed files with 22 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

17
doc/manual/build-farm.xml
View file

@ -31,6 +31,7 @@ variable</link>.</para>
nix@mcflurry.labs.cs.uu.nl powerpc-darwin /home/nix/.ssh/id_quarterpounder_auto 2 nix@mcflurry.labs.cs.uu.nl powerpc-darwin /home/nix/.ssh/id_quarterpounder_auto 2
nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm
nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2 nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2
nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2 kvm perf
</programlisting> </programlisting>
</example> </example>
@ -73,11 +74,12 @@ bits of information:
the machine. If there are multiple machines of the right type, Nix the machine. If there are multiple machines of the right type, Nix
will prefer the fastest, taking load into account.</para></listitem> will prefer the fastest, taking load into account.</para></listitem>
<listitem><para>A comma-separated list of <listitem><para>A comma-separated list of <emphasis>supported
<emphasis>features</emphasis>. If a derivation has the features</emphasis>. If a derivation has the
<varname>requiredSystemFeatures</varname> attribute, then <varname>requiredSystemFeatures</varname> attribute, then
<filename>build-remote.pl</filename> will only perform the <filename>build-remote.pl</filename> will only perform the
derivation on a machine that has the specified features. For instance, the attribute derivation on a machine that has the specified features. For
instance, the attribute
<programlisting> <programlisting>
requiredSystemFeatures = [ "kvm" ]; requiredSystemFeatures = [ "kvm" ];
@ -87,6 +89,15 @@ requiredSystemFeatures = [ "kvm" ];
<literal>kvm</literal> feature (i.e., <literal>scratchy</literal> in <literal>kvm</literal> feature (i.e., <literal>scratchy</literal> in
the example above).</para></listitem> the example above).</para></listitem>
<listitem><para>A comma-separated list of <emphasis>mandatory
features</emphasis>. A machine will only be used to build a
derivation if all of the machine’s mandatory features appear in the
derivation’s <varname>requiredSystemFeatures</varname> attribute.
Thus, in the example, the machine <literal>poochie</literal> will
only do derivations that have
<varname>requiredSystemFeatures</varname> set to <literal>["kvm"
"perf"]</literal> or <literal>["perf"]</literal>.</para></listitem>
</orderedlist> </orderedlist>
You should also set up the environment variable You should also set up the environment variable

12
scripts/build-remote.pl.in
View file

@ -70,13 +70,16 @@ if (defined $conf && -e $conf) {
s/\#.*$//g; s/\#.*$//g;
next if /^\s*$/; next if /^\s*$/;
my @tokens = split /\s/, $_; my @tokens = split /\s/, $_;
my @supportedFeatures = split(/,/, $tokens[5] || "");
my @mandatoryFeatures = split(/,/, $tokens[6] || "");
push @machines, push @machines,
{ hostName => $tokens[0] { hostName => $tokens[0]
, systemTypes => [ split(/,/, $tokens[1]) ] , systemTypes => [ split(/,/, $tokens[1]) ]
, sshKeys => $tokens[2] , sshKeys => $tokens[2]
, maxJobs => int($tokens[3]) , maxJobs => int($tokens[3])
, speedFactor => 1.0 * (defined $tokens[4] ? int($tokens[4]) : 1) , speedFactor => 1.0 * (defined $tokens[4] ? int($tokens[4]) : 1)
, features => [ split(/,/, $tokens[5] || "") ] , supportedFeatures => [ @supportedFeatures, @mandatoryFeatures ]
, mandatoryFeatures => [ @mandatoryFeatures ]
, enabled => 1 , enabled => 1
}; };
} }
@ -90,8 +93,7 @@ my ($drvPath, $hostName, $slotLock);
REQ: while (1) { REQ: while (1) {
$_ = <STDIN> || exit 0; $_ = <STDIN> || exit 0;
my ($amWilling, $neededSystem); (my $amWilling, my $neededSystem, $drvPath, my $requiredFeatures) = split;
($amWilling, $neededSystem, $drvPath, $requiredFeatures) = split;
my @requiredFeatures = split /,/, $requiredFeatures; my @requiredFeatures = split /,/, $requiredFeatures;
my $canBuildLocally = $amWilling && ($localSystem eq $neededSystem); my $canBuildLocally = $amWilling && ($localSystem eq $neededSystem);
@ -117,7 +119,9 @@ REQ: while (1) {
LOOP: foreach my $cur (@machines) { LOOP: foreach my $cur (@machines) {
if ($cur->{enabled} if ($cur->{enabled}
&& (grep { $neededSystem eq $_ } @{$cur->{systemTypes}}) && (grep { $neededSystem eq $_ } @{$cur->{systemTypes}})
&& all(map { my $f = $_; 0 != grep { $f eq $_ } @{$cur->{features}} } @requiredFeatures)) && all(map { my $f = $_; 0 != grep { $f eq $_ } @{$cur->{supportedFeatures}} } (@requiredFeatures, @mandatoryFeatures))
&& all(map { my $f = $_; 0 != grep { $f eq $_ } @requiredFeatures } @{$cur->{mandatoryFeatures}})
)
{ {
$rightType = 1; $rightType = 1;