Connecting Nuget to Gallery Server

Coordinator
Nov 6, 2010 at 4:20 AM

Hi all. I wanted to try connecting Nuget to the gallery server and ran into a few issues I wanted to bring up before filing issues.

1. Ideally, when someone clones the repository, builds the solution, and hits F5 to run it, it should just work. I ran into an issue because Gallery.Server was configured for IIS. It'd be nice if it was configured to use the built in web server during development. Is there some good reason I don't know about that prevents that?

2. I got it running and pointed the NuGet client to the feed (http://localhost:1313/FeedService.svc/) and got an exception in the dialog. Is anyone else experiencing this?

3. One of the features that the gallery should support is the ability to host the actual package file off-site. This is the model that the Web Platform Installer (Web PI) supports. This would allow someone to create an entry in the server gallery for their package, but have the actual package hosted in CodePlex.com. However, to do this safely, we're going to need to know two pieces of information for the package: PackageHash and HashAlgorithm. By default, I think we can make HashAlgorithm always "sha1". The related bug on the Nuget side is: http://nuget.codeplex.com/workitem/18

Developer
Nov 6, 2010 at 7:39 AM

1. This is done deliberately to work around an ugly Cassini bug that makes it unusable with OData (specifically when periods are used, as in our versions)

2. That might be related to the Count issue that Fowler and I saw. I though there was a bug on it but can't find it...

3. Yes, this needs to be supported.  I'll let Kevin comment on that part.

Coordinator
Nov 6, 2010 at 4:38 PM
Thanks David!

1. I can't wait till IIS Express is out. This is a pain since I have to go modify Windows components to install IIS 6 metabase compatibility or whatever the heck that is. :P

2. Ok, what's the workaround in the meanwhile? Change the connection string to point to a real SQL db? Should the project just auto-create a SQL Express db in the meanwhile?


Developer
Nov 6, 2010 at 5:08 PM

1. Indeed, IIS Express is the savior here.

2. Don't know, as I don't think we know much about the issue yet.  David reported it, but it needs some investigation. It would be interesting to see if SQL Express works,, but I don't think anyone has tried yet.

Coordinator
Nov 6, 2010 at 10:29 PM

1) Yes, David is correct. We were originally using Cassini but the OData bug forced us to switch to IIS. Hopefully IIS Express will be the long term answer.

2) Yes, this appears to be a bug translating the Count property somewhere in the stack. I would assume the bug is in the EF code for SQL CE. You can switch to SQL Express, or any full instance of SQL Server, simply by changing the connection string. I've done this and confirmed that it does work - accessing the Count property over OData does not cause an exception like it does with SQL CE.

You can just change the connection in the ConnectionStrings.config file in Gallery.Infrastructure and rebuild (be sure to pull the latest before doing this - I just fixed a problem with the post-build event that copies the ConnectionStrings.config file to the Gallery.Server web project). The file already has a regular SQL Server connection string in it that's commented out that you can use to start with. One caveat with this: EF will attempt to create the database the first time you access the feed. The IIS app will need to have sufficient permissions to do this. I tried using a regular SQL Server account in my connection string instead of integrated security but it didn't work. It appears to be another bug in the EF code-first CTP.

3) The Package entity already has an ExternalPackageUri property for this purpose, although we don't currently have any UI in place for setting this property. One thing we will need to do is implement an API for a client like the gallery or nuget to increment the download count when they download a package from an external source. Currently we increment it automatically when a package is downloaded through the OData feed.

Coordinator
Nov 6, 2010 at 11:43 PM
Thanks Kevin. For #3, it looks like you'll need to add fields for the package hash and the hash algorithm to go along with the ExternalPackageUri.

Perhaps: ExternalPackageHash and ExternalPackageHashAlgorithm


Coordinator
Nov 7, 2010 at 4:02 AM
Edited Nov 7, 2010 at 4:04 AM

Oh, yeah, sorry I forgot to address that. Currently when a package is uploaded to the server we compute the SHA-512 hash and store it with the package metadata. One of the things we talked about for this situation when going over the original spec was that we could simply download the package when they provide an external URL. This would allow us to extract the metadata (including computing the hash) the same way we would if they uploaded the package to the server. The only difference would be that we would not store the physical file on the server, and downloads would then happen from the external source instead of being streamed from the feed. But the submitter would get the same benefit of having the metadata automatically extracted for them so that they don't have to enter everything on the gallery web site.

Coordinator
Nov 8, 2010 at 6:00 AM

I think downloading the package is a good idea.

If we didn’t do that and required users to enter the hash manually, I’d want to suggest changing to SHA1 to make things easier. For example, if I upload my package to Google code, they conveniently show me a SHA1 hash: http://code.google.com/p/subtext/downloads/detail?name=SubtextUpgradeTool.zip&can=2&q=#makechanges

But if we download the package and calculate the hash for them, it doesn’t matter as much.

Phil

Developer
Nov 8, 2010 at 6:05 AM

Yes, downloading the external package was what I assumed all along.  Technically, it's slightly less secure, because there is a small window for the URL to get hacked and the package replace by something malicious, which we would then use as the official hash.  But in practice, this seems like a pretty marginal scenario.  At the time a user submits a package, the assumption is that their external URL is not compromised.

Nov 8, 2010 at 9:22 PM
  • Does NuGet.Core have to support multiple hash algorithms if the hash value is calculated on the server? From the set of algorithms provided by the .NET BCL, SHA512 is the most secure and it would make sense to stick to it. However if we need to support alternate ones, I'd recommend we use naming conventions as specified here:- http://msdn.microsoft.com/en-us/library/aa331332%28v=VS.71%29.aspx
  • How does the gallery plan on keeping track of downloads for externally hosted packages? Would NuGet.Core need to explicitly ping the server to inform it that a package was downloaded / installed?

 

Coordinator
Nov 8, 2010 at 9:57 PM

At minimum, we could get away with only supporting SHA512, but I think we should keep the hashing algorithm field named using the simple name per your suggestion as a means of future-proofing. When the quantum computers beat SHA512, we’ll want the option to change the algorithm while retaining compatibility for existing entries.

Per your second question, I don’t see any other way. Please log a bug on the NuGet side for that. Thanks!