Commit graph

10137 commits

Author SHA1 Message Date
Vincent Ambo
a5473293e7 feat(external): Return references in more useable format for Nix
Sub-packages of external dependencies are traversed by Nix as a tree
of attribute sets which need to be accessed by "path". To make this
easier, the dependency analyser now returns "paths" as string lists.
2019-12-13 00:39:53 +00:00
Vincent Ambo
fb4dd76146 feat(external): Implement tool to analyse external dependencies
Adds a tool that can analyse dependencies that were not originally
meant to be built with buildGo.nix and return information that can be
used to construct appropriate Nix dependencies.

The tool will return information about package-local and foreign
dependencies separately to let Nix determine whether all required
dependencies are provided and to correctly link together sub-packages.

To avoid listing standard library imports in the dependencies, a list
of all packages in the standard library is generated statically to
allow for those to be filtered out during the analysis.

This tool is still work-in-progress.
2019-12-13 00:39:53 +00:00
Vincent Ambo
e60dfabc21 feat(buildGo): Expose Go import path for packages 2019-12-13 00:39:53 +00:00
Vincent Ambo
57f37743c2 fix(buildGo): Ensure 'proto' libraries are overridable 2019-12-13 00:39:53 +00:00
Vincent Ambo
c40e8a4061 feat(buildGo): Add new traversing external' implementation
Adds an alternative implementation of a builder for external packages
which traverses packages and builds up an attribute set tree out of
their structure.

Currently this is not functional because there is no useable method of
specifying dependencies within that package set.
2019-12-13 00:39:53 +00:00
Abseil Team
12bc53e031 Export of internal Abseil changes
--
c99f979ad34f155fbeeea69b88bdc7458d89a21c by Derek Mauro <dmauro@google.com>:

Remove a floating point division by zero test.

This isn't testing behavior related to the library, and MSVC warns
about it in opt mode.

PiperOrigin-RevId: 285220804

--
68b015491f0dbf1ab547994673281abd1f34cd4b by Gennadiy Rozental <rogeeff@google.com>:

This CL introduces following changes to the class FlagImpl:
* We eliminate the CommandLineFlagLocks struct. Instead callback guard and callback function are combined into a single CallbackData struct, while primary data lock is stored separately.
* CallbackData member of class FlagImpl is initially set to be nullptr and is only allocated and initialized when a flag's callback is being set. For most flags we do not pay for the extra space and extra absl::Mutex now.
* Primary data guard is stored in data_guard_ data member. This is a properly aligned character buffer of necessary size. During initialization of the flag we construct absl::Mutex in this space using placement new call.
* We now avoid extra value copy after successful attempt to parse value out of string. Instead we swap flag's current value with tentative value we just produced.

PiperOrigin-RevId: 285132636

--
ed45d118fb818969eb13094cf7827c885dfc562c by Tom Manshreck <shreck@google.com>:

Change null-term* (and nul-term*) to NUL-term* in comments

PiperOrigin-RevId: 285036610

--
729619017944db895ce8d6d29c1995aa2e5628a5 by Derek Mauro <dmauro@google.com>:

Use the Posix implementation of thread identity on MinGW.
Some versions of MinGW suffer from thread_local bugs.

PiperOrigin-RevId: 285022920

--
39a25493503c76885bc3254c28f66a251c5b5bb0 by Greg Falcon <gfalcon@google.com>:

Implementation detail change.

Add further ABSL_NAMESPACE_BEGIN and _END annotation macros to files in Abseil.

PiperOrigin-RevId: 285012012
GitOrigin-RevId: c99f979ad34f155fbeeea69b88bdc7458d89a21c
Change-Id: I4c85d3704e45d11a9ac50d562f39640a6adbedc1
2019-12-12 15:37:13 -05:00
Scott Hart
7abccfa1ec
chore: update googleapis version (#29) 2019-12-11 16:18:51 -05:00
Abseil Team
1e39f8626a Export of internal Abseil changes
--
b842b7fd9b1289be31f0b7ee8e62e48e550747cf by Greg Falcon <gfalcon@google.com>:

Change the Cord str_format formatter to use iteration instead of CordReader.

When Cord is publicly released, CordReader is not going with it.

PiperOrigin-RevId: 284780736

--
28e76c08ea7185a7ff9f4e0e02ae565fbbf7980f by Greg Falcon <gfalcon@google.com>:

Implementation detail change.

Introduce ABSL_NAMESPACE_BEGIN and _END annotation macros which indicate the beginning and end of a `namespace absl` scope.  Currently these do nothing, but they will be used to inject an inline namespace for LTS builds (to avoid symbol collisions against other Abseil versions).

These macros should not be used by end users, because end users should never write `namespace absl {` in their own code.

This CL applies these annotations to all code under //absl/base/.  The rest of Abseil will be annotated in this way in follow-up CLs.

PiperOrigin-RevId: 284776410

--
e1711dc6d696dcca50d4e7d4b4d8f3076575b7ec by Abseil Team <absl-team@google.com>:

--help changed to report long flags.

PiperOrigin-RevId: 284757720

--
78f66a68f428bbbd19d8d60e1125f43ba765fd35 by Tom Manshreck <shreck@google.com>:

Update comment on + or - in SimpleAToi()

PiperOrigin-RevId: 284231843
GitOrigin-RevId: b842b7fd9b1289be31f0b7ee8e62e48e550747cf
Change-Id: I3046b31391bd11c8bc4abab7785a863c377cd757
2019-12-10 15:07:05 -05:00
Vincent Ambo
050b3aabaf fix(read-tree): Only add marker to imported attribute sets
Sometimes things that get imported are (intentionally) not attribute
sets, e.g. for build functions.

Those should not be merged with the marker because, well, that's not
possible.
2019-12-10 14:46:05 +00:00
Vincent Ambo
c93130c8c3 feat(aoc2019): Solve day 3 puzzles
Now this was quite something.
2019-12-10 14:45:28 +00:00
Vincent Ambo
da64d852ea refactor(read-tree): Simplify tree recursion logic
Rewrites the previous initial tick-tocking recursion into a more
straightforward style.

Every attribute set that is imported by readTree now also contains an
attribute called `__readTree` set to `true` which acts as a marker for
other types of tree traversals.

Unfortunately directories without any children or importable content
still result in empty attribute sets, but overall this might be the
better tradeoff vs. having to follow the recursion all the way at each
subtree level to determine which children exist.
2019-12-09 23:23:13 +00:00
Vincent Ambo
2b6b76570e feat(read-tree): Mark derivations that have children
Adds a `__treeChildren` attribute that is set to `true` on derivations
that have children, e.g. for cases where a folder contains a
`default.nix` but has subdirectories with additional things in them.
2019-12-09 19:51:15 +00:00
Vincent Ambo
c81be2f836 feat(aoc2019): Solve day 2 puzzles 2019-12-09 14:43:09 +00:00
Vincent Ambo
85fa12da33 feat(aoc2019): Solve day 1 puzzles 2019-12-09 14:43:01 +00:00
Vincent Ambo
ded10fbd83 feat(tools): Add auto-dispatch for Advent of Code solutions 2019-12-09 12:27:07 +00:00
Vincent Ambo
9fb24840ab chore: Remove old 'external' folder
What was the point of this? Noone knows!
2019-12-09 12:26:45 +00:00
Vincent Ambo
124e185628 fix(read-tree): Always import with locatedAt attribute 2019-12-09 12:22:33 +00:00
Vincent Ambo
514c48fdde feat(tools): Set up Advent of Code 2019 scaffolding 2019-12-09 11:44:21 +00:00
Vincent Ambo
e9431682b2 refactor: Move CI setup to separate Nix file
Instead of polluting the repository namespace with the list of CI
projects, move that to a separate file.

Currently the list of projects to be built by CI is still hardcoded,
but this will be fixed soon.
2019-12-09 10:58:09 +00:00
Vincent Ambo
a1ac598fa0 fix(overrides): Fix package importing for kontemplate 2019-12-09 04:27:40 +00:00
Vincent Ambo
b554b6274b fix(build): Install cachix from third_party 2019-12-09 03:46:19 +00:00
Vincent Ambo
7da979124f fix(third_party): Expose various other missing dependencies 2019-12-09 03:36:50 +00:00
Vincent Ambo
998a2ceb17 fix(third_party): Expose missing dependencies for Nixery 2019-12-09 03:19:03 +00:00
Vincent Ambo
97be9c1411 fix(third_party): Shuffle code around to be compatible with readTree
As per the previous commit these files were no longer being imported,
which is now fixed by moving them into separate subfolders.
2019-12-09 03:18:27 +00:00
Vincent Ambo
ff38ae6c5f fix(read-tree): Don't import default.nix sibling expression
If a folder contains a `default.nix`, Nix expressions contained in
adjacent files should not be imported (they might be things like a
`shell.nix` or a `deps.nix` which do not evaluate to derivations).

The tree traversal still continues for all children folders of a
folder with a `default.nix`.
2019-12-09 03:16:02 +00:00
Vincent Ambo
7044424f52 fix(writeElispBin): Add required nixpkgs attributes for elisp 2019-12-09 03:00:56 +00:00
Vincent Ambo
46e5a23937 fix: Amend import paths to use whitelisted nixpkgs 2019-12-09 02:55:22 +00:00
Vincent Ambo
39ba761663 feat(third_party): Explicitly expose packages from nixpkgs
Instead of exposing the entire package tree from nixpkgs, whitelist
individual packages explicitly so that they show up in
`pkgs.third_party`.

This makes it much easier to control external dependencies used by my
projects.

Bonus: It even includes a working `third_party.callPackage` with only
the whitelisted packages!
2019-12-09 02:55:04 +00:00
Vincent Ambo
77c64ed8f6 fix(build): Temporarily hardcode list of CI projects
Due to a missing feature in readTree I can't currently generate this
list automatically.
2019-12-09 02:52:11 +00:00
Vincent Ambo
01bad09eed refactor: Introduce new layout with nixpkgs in third_party
This is not the final layout yet, but makes it so that my top-level
attribute set is no longer overlaid into nixpkgs itself.

This is useful for other people who are importing my monorepo.
2019-12-09 02:40:48 +00:00
Vincent Ambo
688233acac feat: Add writeElispBin helper function
Exactly as the name suggests. Currently, passing arguments from the
CLI is not really supported.
2019-12-09 02:20:02 +00:00
Vincent Ambo
608a560ee8 refactor(read-tree): Remove deprecated 'builtins.toPath' call 2019-12-09 02:19:44 +00:00
Vincent Ambo
6a526620e2 feat(buildGo): Add 'overrideGo' argument overriding function
This makes it possible to override arguments to the Go builders
downstream in the style of `overrideAttrs` from standard nixpkgs
derivations.

For example, given a Nix value `foo` that builds a binary called `foo`
the name of this binary could be changed and a new dependency on
`somelib` added like so:

    foo.overrideGo(old: {
      name = "bar";
      deps = old.deps ++ [ somelib ];
    })
2019-12-09 01:07:21 +00:00
Abseil Team
77f87009a3 Export of internal Abseil changes
--
d8ddeda8e52132b908fae89b25f117a055d78c04 by Abseil Team <absl-team@google.com>:

Improve performance of ByteStringFromAscii by changing kHexValue to have -1 in invalid value slots. This way a single load can do both the validation and conversion.

PiperOrigin-RevId: 284167344

--
5037e97e2eaaac8ced9a5290949deda4b43b9ceb by Mark Barolak <mbar@google.com>:

Change the underlying symbol name of Cord to absl::Cord.

PiperOrigin-RevId: 284005429

--
4ef66c72aedf135f2b4fd0ba7a73de6642decfff by Abseil Team <absl-team@google.com>:

Eliminate an unnecessary load when futex is contended in Waiter::Wait()

The first argument to compare_exchange_weak() is a reference and will be
updated upon failure.  There is no need to do an additional load on the
same variable.

PiperOrigin-RevId: 284002752
GitOrigin-RevId: d8ddeda8e52132b908fae89b25f117a055d78c04
Change-Id: Idac68a1901eb8c30050adc3860765b1a6fa085c7
2019-12-06 10:28:07 -05:00
Abseil Team
d659fe54b3 Export of internal Abseil changes
--
c385118b3ef0528d150bfe7aeeb63e77f9e463cd by Matt Calabrese <calabrese@google.com>:

Internal-only Archetype generation for testing generic code with user-defined types of various properties.

PiperOrigin-RevId: 283833099

--
4ccf340d3b295aa5b796ee5c97128b61d38899ea by Derek Mauro <dmauro@google.com>:

Fixes the flags parse_test.
Windows doesn't like consecutive path separators.

PiperOrigin-RevId: 283614649

--
5df6d83acb1e49cd1da785cfaf7551f05149f3c9 by Andy Getzendanner <durandal@google.com>:

ABSL_INTERNAL_LOG: forward complete __FILE__ to internal_log_function; not just basename.

PiperOrigin-RevId: 283406080
GitOrigin-RevId: c385118b3ef0528d150bfe7aeeb63e77f9e463cd
Change-Id: Ib0782354691a73fc40185c3262cfd507085b3393
2019-12-05 11:46:50 -05:00
Abseil Team
a4b757b5d4 Export of internal Abseil changes
--
693f81830b9f9cc8b24a1f38492b8dfcdd1d0e24 by Abseil Team <absl-team@google.com>:

Check that absl::int128 works as a std::chrono::duration::rep.

In particular, validate that ...
  std::chrono::time_point<std::chrono::system_clock,
                          std::chrono::duration<absl::int128,
                                                std::atto>>
is a superset (range and resolution) of absl::Time.

PiperOrigin-RevId: 283370280

--
df6073b686bd44223c6f9070fcceec918c728871 by Gennadiy Rozental <rogeeff@google.com>:

Changes thread annotations to use DataGuard() function instead of a specific Mutex.
Remove unused declaration of InvokeCallback.

PiperOrigin-RevId: 283361188

--
b49eb2dd2ee1a0b4c8a7bb1a94e368b81ce5f861 by Abseil Team <absl-team@google.com>:

Rewrite GetNominalCPUFrequency to use advapi32 instead of shlwapi

Using shlwapi.dll means that gdi32.dll is loaded which then makes process destruction more expensive, which is unacceptable for some uses. There may be other places that pull in gdi32.dll - this just fixes the one.

PiperOrigin-RevId: 282960698

--
b5508afec5099a0fdbb55e39a7cd2993259ed860 by Abseil Team <absl-team@google.com>:

Small typo fix in comments: initiazliation -> initialization

PiperOrigin-RevId: 282891800

--
4319cc419584e91ee74f6ae1a32d88a412fc5c01 by Abseil Team <absl-team@google.com>:

Update c_find_first_of() comment to remove the mention of an ordered container.

PiperOrigin-RevId: 282836540

--
5fcabc0a834dff39a505d5a5fc5403ddeb96028e by Derek Mauro <dmauro@google.com>:

Fix NaCl build, where format checking is broken

PiperOrigin-RevId: 282826202

--
aaf9ad3274c056a2f68e9b8ccada45c9802e2f1e by Derek Mauro <dmauro@google.com>:

Fix more -Wundef warnings

PiperOrigin-RevId: 282799820

--
1fb06150a70ffe98bf4b2d42b2a39d083bf44f8c by Derek Mauro <dmauro@google.com>:

Release support for additional platforms

PiperOrigin-RevId: 282793384

--
fa947fc28624a316fa872d7045b3838b88a0d69b by Derek Mauro <dmauro@google.com>:

Cleanup inconsistent usage of __has_attribute

PiperOrigin-RevId: 282793296

--
990030ad282263d6303c83b780a55fdec8e90d43 by Gennadiy Rozental <rogeeff@google.com>:

Eliminate the pointer in absl::Flag, which points to n space where we were storing flag's default value. We also eliminate additional (now unnecessary) allocation for flag's default value.
Instead we'll initialize the flags value directly from the value specified in ABSL_FLAG.
If the default value is updated via the call to SetCommandLineOptionWithMode we are replacing pointer to initialization routine to pointer to new default value.

PiperOrigin-RevId: 282637616
GitOrigin-RevId: 693f81830b9f9cc8b24a1f38492b8dfcdd1d0e24
Change-Id: I6f2edd8ef844de09aa2c182a7ca3133a22364792
2019-12-02 15:53:43 -05:00
Vincent Ambo
d23716f354 refactor(read-tree): Keep traversing even if a default.nix is found
This makes it possible for people to drop a default.nix into folders
along the way that add additional things into the attribute set at
that level.

These default.nix files are imported and merged with the rest of the
traversal from that point on. In theory nothing stops a user from
putting a derivation into one of them, but the effects of merging that
derivation's underlying attribute set with random other things from
the traversal are undefined.

This feature is being introduced for a slight revamp of the thirdParty
layout.
2019-12-02 15:05:54 +00:00
Vincent Ambo
9d6792609f
Merge pull request #10 from tazjin/feat/buildGo-dot-nix
Introduce Bazel-style Nix build system for Go
2019-11-27 15:47:52 +00:00
Vincent Ambo
0bd085f5d6 chore(overrides): Import buildGo.nix from external repository 2019-11-27 15:20:24 +00:00
Vincent Ambo
28e587b348 docs: Add README file that describes project usage & background 2019-11-27 15:14:43 +00:00
Abseil Team
0514227d25 Export of internal Abseil changes
--
2ba0e41a21fbdab36b2f4f3b0dd4b112bd788604 by Derek Mauro <dmauro@google.com>:

Remove the include of <intsafe.h>, which is missing on
some versions of MinGW. DWORD is easily replaced by uint32_t.

PiperOrigin-RevId: 282576177

--
238fd41114b3e83fcb91d2afe1e6dcce7cfd53b0 by Samuel Benzaquen <sbenza@google.com>:

Remove assertion in erase(iterator) that tries to use the comparator.
Add missing this-> qualifier.
Fix bug where node elements are not being destroyed properly.

PiperOrigin-RevId: 282427096

--
6b9446e3b38ed97451c010933e86a572ab659ab2 by Derek Mauro <dmauro@google.com>:

Improves/fixes feature detection in thread_identity

Only use ABSL_PER_THREAD_TLS_KEYWORD when it is supported (previously
on some platforms it evaluated to nothing, which completely breaks
everything), but prefer it to thread_local since benchmarks indicate
it is slightly faster in this critical code path.

Disable the calls to pthread_sigmask on MinGW where it is not
supported.

PiperOrigin-RevId: 282425291
GitOrigin-RevId: 2ba0e41a21fbdab36b2f4f3b0dd4b112bd788604
Change-Id: I34073ecbb4a43ad71f54161c136d88fc728888f1
2019-11-26 12:37:42 -05:00
Vincent Ambo
9280cf9715 feat(buildGo): Add support for gRPC packages
Introduces buildGo.grpc which is like buildGo.proto, but adds
dependencies on the gRPC libraries.
2019-11-26 12:28:10 +00:00
Vincent Ambo
d3e8774a8e feat(proto): Add protobuf & gRPC dependencies via external
Moves the Protobuf & gRPC dependencies to a separate file which uses
buildGo.external to build the dependencies.

The versions are pinned at master of 2019-11-26.
2019-11-26 12:28:10 +00:00
Vincent Ambo
8b6b08b814 feat(buildGo): Add 'srcOnly' and 'targets' parameters for external
Adds two new parameters to buildGo.external:

* `srcOnly` toggles whether the created derivation should contain only
  the source code, or the built package.

  This is useful in situations where some sub-packages of a larger
  package are needed and the build should be deferred to the package
  depending on them.

  It defaults to false, meaning that external packages are built by
  default.

* `targets` controls which "sub-packages" of the target package are
  built. It defaults to building all sub-packages.
2019-11-26 12:21:58 +00:00
Abseil Team
7f4fe64af8 Export of internal Abseil changes
--
44efc1bb0e0a47eabf0569eaab81c66710d5b9c3 by Mark Barolak <mbar@google.com>:

Update "strings::Substitute" to "absl::Substitute" in the absl::Substitute error messages.

PiperOrigin-RevId: 282388042

--
9ec7e9385f5469473f76857dc5b067d869bbc65b by Abseil Team <absl-team@google.com>:

Remove deprecated ExponentialBiased::Get()

PiperOrigin-RevId: 282045123
GitOrigin-RevId: 44efc1bb0e0a47eabf0569eaab81c66710d5b9c3
Change-Id: I915bf0ff5fa7ac2bd5f9fb653d1fbd9ece6af9fc
2019-11-25 14:57:06 -05:00
Vincent Ambo
7d26550c11 feat(buildGo): Add support for building "external" Go libraries
Adds a buildGo.external function that can build packages following the
default go-tool package layout. Dependencies work the same way as they
do for other buildGo-packages, but instead of being passed straight to
the compiler a fake GOPATH is assembled using a symlink forest.

External currently supports very few direct configuration options and
was primarily created to build the protobuf packages, but it is also
useful for including external dependencies in buildGo-native projects.

The previous complex build logic for the protobuf package has been
replaced with a call to `external`.
2019-11-25 16:45:21 +00:00
Vincent Ambo
580bd88622 feat(buildGo): Add x_defs support 2019-11-25 15:34:59 +00:00
Vincent Ambo
43f91c44bd feat(read-tree): Pass in-tree location to imported package sets
Passes the location from the root at which packages are imported on to
all packages.

The path is passed in as a parameter called 'locatedAt' which contains
a list of strings with each individual path component.

For example, the blog source in `services/tazblog` will have a list
with `[ "services" "tazblog" ]` passed in as the `locatedAt`
attribute.

This can be used for enabling features such as path-specific imports
when using things like buildGo.
2019-11-25 15:32:02 +00:00
Vincent Ambo
48ed487bdd refactor(read-tree): Idiomatic 'attrsToList' instead of weird hack 2019-11-25 15:10:31 +00:00
Vincent Ambo
473421dbb8 feat(example): Demonstrate usage of x_defs flag to buildGo.program 2019-11-24 20:41:29 +00:00