Can't install packages under Windows behind a proxy

[haltux]

I am struggling to install packages behind a proxy. I managed to make git working using:

run(`git config --global url."https://".insteadOf git://`)
run(`git config --global http.proxy XX:3128`)

but at some point Julia is using curl to download dependancies (cairo, tk...) and fails. I found no way to set a proxy in curl (I tried _curlrc without success, maybe I don't put it at the right place).

A more serious problem is that each failure leads to an inconsistant state, where no installation attempt is possible anymore, that I managed to solve only by completely removing the .julia folder.

[haltux]

OK, I realized that BinDeps is an external package and it seems that these problems are more BinDeps related than core-Julia related. So I am not sure whether this bug report really belongs here.

What is a bit disturbing is that BinDeps README states that: "The only reason for its early release is its usage for installing Windows binaries for Cairo/Tk during the MIT IAP 2013 Julia tutorial", but it is running by default when installing packages on the standard Julia windows distribution.

[ViralBShah]

The README is a bit out of date. BinDeps has received updates since, and @loladiro should be able to get back into it soon.

[Keno]

I'm planning to give BinDeps a major rewrite just as soon as I'm done with finals.

[espdev]

This problem is of a particular importance. I cannot work in Windows with the package manager behind a proxy.

               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" to list help topics
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.2.0-rc1
 _/ |\__'_|_|_|\__'_|  |  Commit 040f3ab 2013-10-15 05:58:39 UTC
|__/                   |  x86_64-w64-mingw32

julia> Pkg.status()
INFO: Initializing package repository U:\.julia
INFO: Cloning METADATA from git://github.com/JuliaLang/METADATA.jl
fatal: unable to connect to github.com:
github.com[0: 192.30.252.130]: errno=No error

ERROR: failed process: Process(`git clone -q -b metadata-v2 git://github.com/Jul
iaLang/METADATA.jl METADATA`, ProcessExited(128)) [128]
 in pipeline_error at process.jl:447
 in run at process.jl:424
 in info at error.jl:68
WARNING: backtraces on your platform are often misleading or partially incorrect

[StefanKarpinski]

Have you tried the insteadOf thing?

[quinnj]

@haltux, @espdev, are you guys still seeing issues here? In particular, we usually encourage the use of the following git idiom to use https instead of the git protocol. Let us know if you're still seeing the issue and if this works to get around.

If you are behind a firewall and you need to use the https protocol instead of the git protocol:

git config --global url."https://".insteadOf git://

[quinnj]

Feel free to reopen if you're still seeing issues.

[tknopp]

I know that this issue is well documented but still I wonder if there is any automatic fallback. This issue pops up regularly when I introduce Julia to new users and most of the time they do not find the workaround. Maybe when Pkg.add fails one should just have a verbose text that suggest to execute the insteadOf command.

[tkelman]

I imagine the recommended workaround will need to be adjusted when command-line git gets replaced by libgit2 anyway. We'll have to see whether and how to address the problem, in documentation and/or as an automatic fallback, once Pkg is libgit2-based.

[tknopp]

Fair enough. Just wanted to give feedback where I found several newcomers struggle with.

[tkelman]

We do have one other example of modifying the default git config in the bundled command-line git on the 0.3 series of Windows binaries (for line endings), if this would be a good thing to add by default for everyone we could try that.

[djpeaco]

I believe that this is still a problem. I just tested version 0.3.4 and the nightly build behind our corporate firewall/proxy, and am getting errors installing some packages. For example, when running Pkg.add("IJulia"), I get the following error:

INFO: Downloading file http://s3.amazonaws.com/julialang/bin/winnt/extras/libzmq-3.3-x64.zip
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                               Dload  Upload   Total   Spent    Left  Speed
0     0    0     0    0     0      0      0 --:--:--  0:00:20 --:--:--     0
curl: (7) Failed connect to s3.amazonaws.com:80; No error
=================================[ ERROR: ZMQ ]=================================
failed process: Process(`curl -o 'C:\Users\djpeaco\.julia\v0.3\ZMQ\deps\downloads\libzmq-3.3-x64.zip' -L http://s3.amazonaws.com/julialang/bin/winnt/extras/libzmq-3.3-x64.zip`, ProcessExited(7)) [7]
while loading C:\Users\djpeaco\.julia\v0.3\ZMQ\deps\build.jl, in expression starting on line 21
===============================================================================

To reiterate what haltux said, the problem is occurring with curl, not git; all the git clones are working fine. I've tried the git insteadOf work-around and it doesn't fix this particular problem. Other packages, like Gadfly, install just fine.

[tkelman]

@djpeaco out of curiosity, does the following work? Pkg.add("WinRPM"); using WinRPM; WinRPM.install("zeromq") ?

I think WinRPM is directly using the Windows API for downloading files here, which BinDeps should probably do too instead of shelling out to the version of curl that we happen to bundle along with command-line git (which we plan on getting rid of at some point).

[djpeaco]

Yes, WinRPM.install("zeromq") runs successfully on the nightly build (on 0.3.4 I get error code 2148270086 trying to download libgcc_s_seh1, which the nightly build doesn't attempt to download/install).

[tkelman]

I get error code 2148270086 trying to download libgcc_s_seh1, which the nightly build doesn't attempt to download/install

That should be happening in the package, not base Julia, so it wouldn't be any different. The only difference would be if you've already run that part before on your nightly installation. That download problem is hopefully spurious, if you try again a few times I hope one would eventually go through (we're hoping to make this more reliable, we need some advice from someone who understands how to mirror an rpm repository first).

Can you open a couple of issues for the ZMQ.jl and BinDeps.jl packages? cc me with @tkelman on each of them. ZMQ.jl should switch to using a WinRPM binary, and BinDeps.jl should stop trying to use curl to download files on Windows.