Commit graph

9903 commits

Author SHA1 Message Date
Vincent Ambo
9594203e07 feat(third_party/gopkgs): Import external packages for Google Cloud
These packages contain the Cloud SDK for Go. There is currently a
linker issue (presumably due to something in `buildGo.nix`) that means
that projects using them can not actually be built.
2019-12-13 12:12:06 +00:00
Vincent Ambo
8b10c8d0f3 chore(overrides): Upgrade buildGo.nix 2019-12-13 12:11:50 +00:00
Vincent Ambo
5649c75d01 fix(external): Fix "inverted" local dependencies
Usually in large packages the root depends on one or more
sub-packages (or there is no root), but some projects (e.g.
golang.org/x/oauth2) do it the other way around.

This fix adds compatibility for both ways.
2019-12-13 12:10:32 +00:00
Vincent Ambo
9d5417501b docs(README): Update description of buildGo.external 2019-12-13 00:39:53 +00:00
Vincent Ambo
369c86e0ef fix(proto): Refactor gRPC dependencies to match new external layout
This is now a lot more fine-grained than before, but it actually works
fine.

This stuff is a bit annoying to write by hand. There are multiple
different options available (e.g. carrying an attribute set of all
subpackages in each `external` and only passing that, having a tool
generate this, etc.).
2019-12-13 00:39:53 +00:00
Vincent Ambo
41c28cf60e fix(buildGo): Correctly refer to goGrpc package 2019-12-13 00:39:53 +00:00
Vincent Ambo
d8ef203f78 feat(external): Support foreign dependencies in external packages
Users can supply a list of foreign dependencies in calls to
buildGo.external. These are now appropriately inserted into packages
that *need them* and no further, resolving issues with complex
internal recursion in some repositories!
2019-12-13 00:39:53 +00:00
Vincent Ambo
218b81c709 fix(external): Skip folders with "no buildable Go files"
This error is returned by the build analysis logic if the target
package does not have any relevant Go files, which might be the case
if `+build` flags and such are used.
2019-12-13 00:39:53 +00:00
Vincent Ambo
39e73f42b2 style(external): Minor formatting fixes 2019-12-13 00:39:53 +00:00
Vincent Ambo
0dfa5d15fa fix(buildGo): Update buildGo.proto for compatibility with external
Changes in the structure of buildGo.external meant that the package
layout for the protobuf library is now slightly different.

`proto` has been amended to work with the new structure. Callers of
buildGo.proto do not need to be updated (i.e. the example still
works).
2019-12-13 00:39:53 +00:00
Vincent Ambo
875628a097 feat(external): Switch between packages & programs automatically 2019-12-13 00:39:53 +00:00
Vincent Ambo
f5e3183de1 feat(buildGo): Expose new external builder 2019-12-13 00:39:53 +00:00
Vincent Ambo
c5373a69fe feat(external): Implement builder function for externals
Implements a builder function that calls the analysis tool on the
provided source and builds up the required attribute set, including
local dependencies.
2019-12-13 00:39:53 +00:00
Vincent Ambo
bbf3a418a5 feat(external): Add fully qualified import path to analyser output
This is used by Nix to build the derivation names for individual
packages.
2019-12-13 00:39:53 +00:00
Vincent Ambo
1fd80fb201 fix(external): Correctly set names for root packages
Fixes the prefix trimming logic for package names and source files if
the source files appear in the package root (which is, unsurprisingly,
very common).
2019-12-13 00:39:53 +00:00
Vincent Ambo
b20e46d60b fix(external): Ensure findGoDirs "finds" top-level directory
Due to the lexical walk order of `filepath.Walk` the previous
directory identification logic failed under certain conditions if the
top-level directory contained Go files that showed up *after* the
first subdirectories.

To simplify the logic a set of directories is now gathered instead on
a file-level.
2019-12-13 00:39:53 +00:00
Vincent Ambo
a5473293e7 feat(external): Return references in more useable format for Nix
Sub-packages of external dependencies are traversed by Nix as a tree
of attribute sets which need to be accessed by "path". To make this
easier, the dependency analyser now returns "paths" as string lists.
2019-12-13 00:39:53 +00:00
Vincent Ambo
fb4dd76146 feat(external): Implement tool to analyse external dependencies
Adds a tool that can analyse dependencies that were not originally
meant to be built with buildGo.nix and return information that can be
used to construct appropriate Nix dependencies.

The tool will return information about package-local and foreign
dependencies separately to let Nix determine whether all required
dependencies are provided and to correctly link together sub-packages.

To avoid listing standard library imports in the dependencies, a list
of all packages in the standard library is generated statically to
allow for those to be filtered out during the analysis.

This tool is still work-in-progress.
2019-12-13 00:39:53 +00:00
Vincent Ambo
e60dfabc21 feat(buildGo): Expose Go import path for packages 2019-12-13 00:39:53 +00:00
Vincent Ambo
57f37743c2 fix(buildGo): Ensure 'proto' libraries are overridable 2019-12-13 00:39:53 +00:00
Vincent Ambo
c40e8a4061 feat(buildGo): Add new traversing external' implementation
Adds an alternative implementation of a builder for external packages
which traverses packages and builds up an attribute set tree out of
their structure.

Currently this is not functional because there is no useable method of
specifying dependencies within that package set.
2019-12-13 00:39:53 +00:00
Abseil Team
12bc53e031 Export of internal Abseil changes
--
c99f979ad34f155fbeeea69b88bdc7458d89a21c by Derek Mauro <dmauro@google.com>:

Remove a floating point division by zero test.

This isn't testing behavior related to the library, and MSVC warns
about it in opt mode.

PiperOrigin-RevId: 285220804

--
68b015491f0dbf1ab547994673281abd1f34cd4b by Gennadiy Rozental <rogeeff@google.com>:

This CL introduces following changes to the class FlagImpl:
* We eliminate the CommandLineFlagLocks struct. Instead callback guard and callback function are combined into a single CallbackData struct, while primary data lock is stored separately.
* CallbackData member of class FlagImpl is initially set to be nullptr and is only allocated and initialized when a flag's callback is being set. For most flags we do not pay for the extra space and extra absl::Mutex now.
* Primary data guard is stored in data_guard_ data member. This is a properly aligned character buffer of necessary size. During initialization of the flag we construct absl::Mutex in this space using placement new call.
* We now avoid extra value copy after successful attempt to parse value out of string. Instead we swap flag's current value with tentative value we just produced.

PiperOrigin-RevId: 285132636

--
ed45d118fb818969eb13094cf7827c885dfc562c by Tom Manshreck <shreck@google.com>:

Change null-term* (and nul-term*) to NUL-term* in comments

PiperOrigin-RevId: 285036610

--
729619017944db895ce8d6d29c1995aa2e5628a5 by Derek Mauro <dmauro@google.com>:

Use the Posix implementation of thread identity on MinGW.
Some versions of MinGW suffer from thread_local bugs.

PiperOrigin-RevId: 285022920

--
39a25493503c76885bc3254c28f66a251c5b5bb0 by Greg Falcon <gfalcon@google.com>:

Implementation detail change.

Add further ABSL_NAMESPACE_BEGIN and _END annotation macros to files in Abseil.

PiperOrigin-RevId: 285012012
GitOrigin-RevId: c99f979ad34f155fbeeea69b88bdc7458d89a21c
Change-Id: I4c85d3704e45d11a9ac50d562f39640a6adbedc1
2019-12-12 15:37:13 -05:00
Scott Hart
7abccfa1ec
chore: update googleapis version (#29) 2019-12-11 16:18:51 -05:00
Abseil Team
1e39f8626a Export of internal Abseil changes
--
b842b7fd9b1289be31f0b7ee8e62e48e550747cf by Greg Falcon <gfalcon@google.com>:

Change the Cord str_format formatter to use iteration instead of CordReader.

When Cord is publicly released, CordReader is not going with it.

PiperOrigin-RevId: 284780736

--
28e76c08ea7185a7ff9f4e0e02ae565fbbf7980f by Greg Falcon <gfalcon@google.com>:

Implementation detail change.

Introduce ABSL_NAMESPACE_BEGIN and _END annotation macros which indicate the beginning and end of a `namespace absl` scope.  Currently these do nothing, but they will be used to inject an inline namespace for LTS builds (to avoid symbol collisions against other Abseil versions).

These macros should not be used by end users, because end users should never write `namespace absl {` in their own code.

This CL applies these annotations to all code under //absl/base/.  The rest of Abseil will be annotated in this way in follow-up CLs.

PiperOrigin-RevId: 284776410

--
e1711dc6d696dcca50d4e7d4b4d8f3076575b7ec by Abseil Team <absl-team@google.com>:

--help changed to report long flags.

PiperOrigin-RevId: 284757720

--
78f66a68f428bbbd19d8d60e1125f43ba765fd35 by Tom Manshreck <shreck@google.com>:

Update comment on + or - in SimpleAToi()

PiperOrigin-RevId: 284231843
GitOrigin-RevId: b842b7fd9b1289be31f0b7ee8e62e48e550747cf
Change-Id: I3046b31391bd11c8bc4abab7785a863c377cd757
2019-12-10 15:07:05 -05:00
Vincent Ambo
050b3aabaf fix(read-tree): Only add marker to imported attribute sets
Sometimes things that get imported are (intentionally) not attribute
sets, e.g. for build functions.

Those should not be merged with the marker because, well, that's not
possible.
2019-12-10 14:46:05 +00:00
Vincent Ambo
c93130c8c3 feat(aoc2019): Solve day 3 puzzles
Now this was quite something.
2019-12-10 14:45:28 +00:00
Vincent Ambo
da64d852ea refactor(read-tree): Simplify tree recursion logic
Rewrites the previous initial tick-tocking recursion into a more
straightforward style.

Every attribute set that is imported by readTree now also contains an
attribute called `__readTree` set to `true` which acts as a marker for
other types of tree traversals.

Unfortunately directories without any children or importable content
still result in empty attribute sets, but overall this might be the
better tradeoff vs. having to follow the recursion all the way at each
subtree level to determine which children exist.
2019-12-09 23:23:13 +00:00
Vincent Ambo
2b6b76570e feat(read-tree): Mark derivations that have children
Adds a `__treeChildren` attribute that is set to `true` on derivations
that have children, e.g. for cases where a folder contains a
`default.nix` but has subdirectories with additional things in them.
2019-12-09 19:51:15 +00:00
Vincent Ambo
c81be2f836 feat(aoc2019): Solve day 2 puzzles 2019-12-09 14:43:09 +00:00
Vincent Ambo
85fa12da33 feat(aoc2019): Solve day 1 puzzles 2019-12-09 14:43:01 +00:00
Vincent Ambo
ded10fbd83 feat(tools): Add auto-dispatch for Advent of Code solutions 2019-12-09 12:27:07 +00:00
Vincent Ambo
9fb24840ab chore: Remove old 'external' folder
What was the point of this? Noone knows!
2019-12-09 12:26:45 +00:00
Vincent Ambo
124e185628 fix(read-tree): Always import with locatedAt attribute 2019-12-09 12:22:33 +00:00
Vincent Ambo
514c48fdde feat(tools): Set up Advent of Code 2019 scaffolding 2019-12-09 11:44:21 +00:00
Vincent Ambo
e9431682b2 refactor: Move CI setup to separate Nix file
Instead of polluting the repository namespace with the list of CI
projects, move that to a separate file.

Currently the list of projects to be built by CI is still hardcoded,
but this will be fixed soon.
2019-12-09 10:58:09 +00:00
Vincent Ambo
a1ac598fa0 fix(overrides): Fix package importing for kontemplate 2019-12-09 04:27:40 +00:00
Vincent Ambo
b554b6274b fix(build): Install cachix from third_party 2019-12-09 03:46:19 +00:00
Vincent Ambo
7da979124f fix(third_party): Expose various other missing dependencies 2019-12-09 03:36:50 +00:00
Vincent Ambo
998a2ceb17 fix(third_party): Expose missing dependencies for Nixery 2019-12-09 03:19:03 +00:00
Vincent Ambo
97be9c1411 fix(third_party): Shuffle code around to be compatible with readTree
As per the previous commit these files were no longer being imported,
which is now fixed by moving them into separate subfolders.
2019-12-09 03:18:27 +00:00
Vincent Ambo
ff38ae6c5f fix(read-tree): Don't import default.nix sibling expression
If a folder contains a `default.nix`, Nix expressions contained in
adjacent files should not be imported (they might be things like a
`shell.nix` or a `deps.nix` which do not evaluate to derivations).

The tree traversal still continues for all children folders of a
folder with a `default.nix`.
2019-12-09 03:16:02 +00:00
Vincent Ambo
7044424f52 fix(writeElispBin): Add required nixpkgs attributes for elisp 2019-12-09 03:00:56 +00:00
Vincent Ambo
46e5a23937 fix: Amend import paths to use whitelisted nixpkgs 2019-12-09 02:55:22 +00:00
Vincent Ambo
39ba761663 feat(third_party): Explicitly expose packages from nixpkgs
Instead of exposing the entire package tree from nixpkgs, whitelist
individual packages explicitly so that they show up in
`pkgs.third_party`.

This makes it much easier to control external dependencies used by my
projects.

Bonus: It even includes a working `third_party.callPackage` with only
the whitelisted packages!
2019-12-09 02:55:04 +00:00
Vincent Ambo
77c64ed8f6 fix(build): Temporarily hardcode list of CI projects
Due to a missing feature in readTree I can't currently generate this
list automatically.
2019-12-09 02:52:11 +00:00
Vincent Ambo
01bad09eed refactor: Introduce new layout with nixpkgs in third_party
This is not the final layout yet, but makes it so that my top-level
attribute set is no longer overlaid into nixpkgs itself.

This is useful for other people who are importing my monorepo.
2019-12-09 02:40:48 +00:00
Vincent Ambo
688233acac feat: Add writeElispBin helper function
Exactly as the name suggests. Currently, passing arguments from the
CLI is not really supported.
2019-12-09 02:20:02 +00:00
Vincent Ambo
608a560ee8 refactor(read-tree): Remove deprecated 'builtins.toPath' call 2019-12-09 02:19:44 +00:00
Vincent Ambo
6a526620e2 feat(buildGo): Add 'overrideGo' argument overriding function
This makes it possible to override arguments to the Go builders
downstream in the style of `overrideAttrs` from standard nixpkgs
derivations.

For example, given a Nix value `foo` that builds a binary called `foo`
the name of this binary could be changed and a new dependency on
`somelib` added like so:

    foo.overrideGo(old: {
      name = "bar";
      deps = old.deps ++ [ somelib ];
    })
2019-12-09 01:07:21 +00:00
Abseil Team
77f87009a3 Export of internal Abseil changes
--
d8ddeda8e52132b908fae89b25f117a055d78c04 by Abseil Team <absl-team@google.com>:

Improve performance of ByteStringFromAscii by changing kHexValue to have -1 in invalid value slots. This way a single load can do both the validation and conversion.

PiperOrigin-RevId: 284167344

--
5037e97e2eaaac8ced9a5290949deda4b43b9ceb by Mark Barolak <mbar@google.com>:

Change the underlying symbol name of Cord to absl::Cord.

PiperOrigin-RevId: 284005429

--
4ef66c72aedf135f2b4fd0ba7a73de6642decfff by Abseil Team <absl-team@google.com>:

Eliminate an unnecessary load when futex is contended in Waiter::Wait()

The first argument to compare_exchange_weak() is a reference and will be
updated upon failure.  There is no need to do an additional load on the
same variable.

PiperOrigin-RevId: 284002752
GitOrigin-RevId: d8ddeda8e52132b908fae89b25f117a055d78c04
Change-Id: Idac68a1901eb8c30050adc3860765b1a6fa085c7
2019-12-06 10:28:07 -05:00