James Manning

These aren’t in the order I learned them.  They’re not in the order of importance, either.  They are in order, though.

• Even if you plan on running it under IIS, do the development with the dev server built-in to VS (Cassini)
  • There’s lots of little problems you may hit running under IIS, so having it working under Cassini first lets you know which problems are caused by IIS config :)
• Cassini (at least in my 2010 RC) only binds to 127.0.0.1 / IPv4 when it does its localhost binding, not ::1 / IPv6
  • 
  • This surfaced when IE could do the GET on the service via localhost but Chrome couldn’t – not sure why, but Chrome was hitting the IPv6 ::1 and IE wasn’t.
  • No big deal, just changed host part of url to 127.0.0.1 to force the issue in both.
• Chrome doesn’t display xml response nicely like IE does – appears to try to render it as html.
  • Content type response header was application/xml; charset=utf-8, so it kind of surprised me that it didn’t render to something friendlier.
  • version of “4.0.249.89 (38071)” (stable branch)
• Under IIS, WebDAV by default includes a lot of verbs that it handles for all extensions. 
  • In inetmgr, Handler mappings -> WebDAV -> Request Restrictions -> Verbs
  • PROPFIND,PROPPATCH,MKCOL,PUT,COPY,DELETE,MOVE,LOCK,UNLOCK
  • Since that list includes PUT and DELETE by default, if GET and POST work fine but those 2 don’t, that’s one thing to check.
  • Simplest fix is to fire up appwiz.cpl and get rid of WebDAV, although you could just remove those 2 verbs from the list if you wanted.  Of course, doing so you’d presumably break WebDAV clients, so I’m not sure that’s of any real value :)
  • 
• Much like WebDAV, the asp.net compat settings, like [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Required)] and <serviceHostingEnvironment aspNetCompatibilityEnabled="true"/>, may break certain verbs like PUT and DELETE for non-obvious reasons.
  • If you don’t need them, don’t use them, of course :)
• You don’t have to use an interface for your service contract, it can be on the real implementation class (tons of examples do this), but if you want to have consumers use WebChannelFactory, you need an interface.
  • On a related note, I apparently wish “where T is an interface” was a viable generic constraint, since that would have been nice in this case instead of hitting it at runtime :)
• I really liked putting the ServiceContract/OperationContract/WebGet/WebInvoke attributes in the interface to make the implementation class a little cleaner.
  • Certainly makes sense to have them there given the client consumption :)
• It makes me sad that UriTemplate only supports string as the type if the template part being matched is a path segment.
  • It can map other types (int, long, etc) if it’s in a query string, but I’d rather do the Convert call myself than make (IMHO) uglier access URI’s :)
• If you’re using something ‘manual’ like Fiddler’s Request Builder for making your REST calls, you actually get a pretty decent error message in the body of the 400 Bad Request response:
  • The incoming message has an unexpected message format ‘Raw’. The expected message formats for the operation are ‘Xml’, ‘Json’. This can be because a WebContentTypeMapper has not been configured on the binding. See the documentation of WebContentTypeMapper for more details.
  • Reminded me to add Content-Type: text/xml and then it was happy :)
• The WCF Test Client that gets run if your svc is set as your startup page will still query for WSDL and complain that it can’t find metadata for your service.
• The WCF REST Starter Kit for 3.5 is great, but it hasn’t been / won’t be updated for 4.0.
  • The one thing it was nice to see that made it back into the framework in 4.0 was automatic generation of a REST service’s help page via <webHttp helpEnabled="true" />
  • Not clear off-hand what the help page lacks for generation of a proxy similar to SOAP/WSDL, but I haven’t dug into it, either.  It sure looks like nicely structured, but perhaps there’s some metadata not getting exposed there.

I have not failed, I’ve just found 10,000 ways that won’t work.

Share:

Based on the last line, I’m really surprised that don’t have error reporting on by default for all pre-RTM builds (although maybe it was on in the beta and they turned it off for the RC).

Checking the compiler options, it looks like I should have gotten a prompt but didn’t.

However, I then ran Fiddler and ran the build again (causing another crash) – yup, the watson reports are getting sent in just fine.  Kinda surprised I didn’t get a prompt, but that may just be a bit they set to always send if it’s a pre-RTM build (which would be reasonable IMHO).

Share:

Until the official Google Calendar Sync supports 2010, you’re stuck either living without it, or using one of the ‘hack’ methods to get it working.

NOTE: this post is more detailed than it needs to be – consider yourself warned :)

The ones that show up time and again on the support forum thread for this are:

1. using a hacked version of GoogleCalendarSync.exe
2. hacking your version of Outlook.exe

Both work fine in my experience, so I’ll cover them both here.  Since some people may be more comfortable with one approach over the other, I’ll let you pick

Hacking GoogleCalendarSync.exe

This is certainly the easier approach of the two.

1. Install Google Calendar Sync the way you normally would: http://dl.google.com/googlecalendarsync/GoogleCalendarSync_Installer.exe
2. Download this zip file: http://mychannellogos.com/Documents/GoogleCalendarSync.zip
3. IMPORTANT: unblock the zip file before extracting the file from it.
   • Go to the folder that contains the zip file after you downloaded it
   • Right-click on the zip file and select Properties (or alt-double-click or alt-enter or whatever)
     • 
   • Click Unblock
     • 
   • Click OK
   • NOTE: if you forget this step – you could always unblock the exe after extraction, but I like to unblock the zip in case I want to re-extract the exe again later on.
4. Extract/copy the GoogleCalenderSync.exe from the zip over to your <Program Files>\Google\Google Calender Sync
   • You probably want to make a backup copy of the existing exe first, just in case, although if you forget and want it back, you can obviously just reinstall it.

One concern people might have is using some random exe off the internet in this way.  Just to check, I did a binary diff.  Well, I tried to use the diff.exe that came with (g)vim, but –binary just said they were different instead of giving me something like a breakdown of byte differences.  Rather than fight with it, I just sucked the 2 exe’s into byte arrays to compare:

[180] C:\Program Files\Google\Google Calendar Sync » $new = gc -encoding byte .\GoogleCalendarSync.exe
[181] C:\Program Files\Google\Google Calendar Sync » $old = gc -encoding byte .\GoogleCalendarSync.exe.orig
[184] C:\Program Files\Google\Google Calendar Sync » compare $old $new

                                                                    InputObject SideIndicator
                                                                    ———– ————-
                                                                            235 =>
                                                                            117 <=

So, hacked version has a byte of 0xEB where the orig has 0×75 – didn’t see an obvious way out of compare to get the offset, so did the obvious/lazy way

[204] C:\Program Files\Google\Google Calendar Sync » foreach ($i in 1 .. $old.length) { if (compare $old[$i] $new[$i]) { $i } }
185732
[205] C:\Program Files\Google\Google Calendar Sync » ‘{0:x}’ -f 185732
2d584

With Notepad++ using the Hex Editor plugin, you can see them with the surrounding bytes:

What does that byte change?  Well, let’s look at it in VS, by starting the orig exe (probably a way to get the disassembly in VS without the running process, but I know this way already :), attaching the debugger, and going to that offset.  Since the exe loads at 0×00400000 (checking Debug –> Windows –> Modules), I need to add the offset to that to get the right memory location of 0×0042D584.

Going to that spot in memory in the disassembly shows it’s a jne instruction:

Setting a breakpoint there and then stepping once, we can see in the orig exe case, the failure case doesn’t take that jump:

If we let it run (F5) from there, we get the error dialog that everyone that’s hit this is familiar with:

So what about the hacked version of the exe?  What does that one byte change do?

You can likely guess it based on the above behavior – it just makes the jump unconditional, ignoring the result of the compare:

So, hopefully that will help clear up any fear that the modified exe is doing something nefarious – it’s no more nefarious than the original exe :)

Hacking Outlook.exe

This one’s a little more difficult than the above, simply because you most likely want to use a hex editor to make this change rather than trying to do so by loading the exe into a normal text editor.  As with the previous approach, you likely want to make a backup copy of the .exe before modifying it.

The file you want to edit is the 2010 version, so make sure to edit the one in <Program Files>\Microsoft Office\Office14 (if you still have other versions installed, you’ll have other outlook.exe files in other folders).

What if you don’t have a hex editor handy but still want to take this approach?  There’s tons of hex editors out there, but since it’s what I happened to pick for this example, I’ll mention the steps for Notepad++

1. Install Notepad++
• Main site: http://notepad-plus.sourceforge.net/
• Files area: http://sourceforge.net/projects/notepad-plus/files/
• IMHO, you’ll want the installer exe – just click the green button here on this page:
• 
2. Install Hex Editor plugin
• Nice thing about Notepad++ is that it has a Plugin Manager that lists both installed and available plugins.
• 
3. The unfortunate thing is that, at least at the moment, the Hex Editor plugin is not considered stable, so to get it to show in the list of Available plugins, you need to click on the Settings button…
• 
4. … and then check “Show unstable plugins”
• 
5. With that checked, HEX-Editor will then show up in the list of available plugins:
• 
6. Check the box next to it, click the Install button, and then let Notepad++ restart when the Plugin Manager prompts you to do so:
• 

For now, I’ll just assume you’re using Notepad++ with the hex editor plugin – as I mentioned, there’s plenty of others, so feel free to use another if you want – the instructions aren’t appreciably different.

Open the Outlook.exe file in Notepad++ (make sure Outlook is not running :) – you can do this in any of the usual ways, including:

• drag-and-drop the file from explorer into the running app
• right-click Outlook.exe and select “Edit with Notepad++”
• File –> Open (or Control+O) and browse around and select the file

When initially opened up, you’re not in the hex view yet, so you get the text editor view with lots of embedded nulls:

Site note: For those who don’t already know – the magic number at the front – the ASCII letters ‘MZ’, are because of Mark Zbikowski

You instead want to look at the hex view of the file, so we select “View in Hex” from the plugin’s submenu:

Thankfully finding the place in the file to modify is pretty simple – just look for the (ANSI/ASCII) string “14.0.0” – it happens to be prefixed with an open paren, but even if you leave that out of the search string, there’s only 1 match (at least in my version, 14.0.4536.1000, 32-bit, but it appears true for other builds based on the forum thread).

In Notepad++, you can bring up the find window with Control+F.  The first thing you’ll want to do is change the data type to “ANSI String”:

Once that’s changed, we can put in the search string of 14.0.0:

Click Find Next and you’ll get to the match (only one in the file, at least in my version):

Change the “14” to “12”.  Easiest way is clicking just before the “4” in the right-hand column and then typing “2” to overwrite it – resulting in:

(ignore the underlined part on the left side – that just shows you the byte that’s just to the right of the cursor – the one you would potentially be overwriting).

As you can see, the byte “34” changed to “32” in the hex view (left side).

Now, just save the file, start Outlook, then tell Google Calendar Sync to go ahead and do a sync now just to make sure it works:

Of course, add an appointment on each side (google calendar and outlook) then do another sync to make sure it copies both ways :)

That’s it – since both ways work for me, hopefully at least one of them will work for you :)

Share:

I was hopping over to the U-Prove CTP link (linked from Channel9 and Ars Technica)

https://connect.microsoft.com/content/content.aspx?contentid=12505&siteid=642

It’s the same url that you get redirected to if you follow the “main link” for U-Prove off of MSCOM at the moment:

http://www.microsoft.com/u-prove

However, it failed to load with what amounts to a permissions error:

Notice in the above that I’m signed in, though.

If I sign out, then the link works fine:

Perhaps my Connect account has been tagged with some kind of “he’s eeeeevil!” bit, but it’s certainly the first time I’ve had more permission on a site being anonymous than logged in :)

Share:

SQL Server Reporting Services keeps a nice view around for looking at the performance stats – ExecutionLog2 (new to 2008, obviously the ExecutionLog view predated it :)

http://blogs.msdn.com/robertbruckner/archive/2009/01/05/executionlog2-view.aspx

Before you can optimize particular reports or your entire system, you need metrics and understand what they tell you.  In this posting, I want to focus on how to effectively interpret and utilize the data present in the new ExecutionLog2 view in the Reporting Services 2008 catalog database.  In summary, I am covering the following topics:

• Description of ExecutionLog2 columns, with tips on how to interpret values
• ExecutionLog2.AdditionalInfo and some interesting pieces of information it provides
• Tips for analyzing ExecutionLog2 information
• Tips for optimizing reports

When you go to query it, just keep in mind that Parameters is an ntext column, so you’ll want to use ‘like’ instead of trying to ‘=’ with a particular string.  Not sure what they didn’t make it nvarchar(max) since that was introduced in 2005, although it may be just to minimize the change between the 2 views.

If you’d used the ExecutionLog view that predates this one, you’re likely to appreciate the case statements they added in the view definition of ExecutionLog2 to change the RequestType and Source to human-readable values :)

Also note that Robert’s blog has tons of good info on SSRS in general and performance in particular – go subscribe :)

Share:

Edit [2010-03-01] – turns out Reflector can’t handle reversing any iterator blocks.  Ouch.

I’m not sure how helpful this post will be for others, but I needed a place to stick screen shots since I can’t attach files to Reflector’s in-app “Send Feedback” dialog nor when creating posts in their forum.

I was decompiling the Google.GData.Client.dll that comes with the (binary-only in the YouTube SDK msi) YouTubeUploader.exe – normally this wouldn’t make much sense, since the source is right here, but ignore that for the purpose of this post, please. :)

Feed<T> in that assembly has a nice Entries property of type IEnumerable<T> (starts @ line 202 in current trunk version of the file) implemented as an iterator block (using “yield return”, “yield break”).  It certainly has more logic than the average iterator block you’ll find out there, so I’m guessing part of the work the compiler had to do when implementing it confused reflector.  Whatever the cause, Reflector doesn’t currently handle correctly reversing it back to the iterator block in the source.

Reflector still shows you the generated code, of course – it just failed in whatever pass it does to identify these bits as the generated pattern and reverse it back.

Here’s the inner class of Feed<T> that’s generated:

And here’s the Entries property:

Most of the heavy lifting is done in the generated inner class’s MoveNext method. (too long and of too little value to put inline :)

If the above looks bizarre and/or you’re still curious about how the iterator pattern is implemented by the compiler (as state machines), read more here:

• Jon’s excellent article on iterator blocks
• Eric’s series of posts on iterator blocks
  • Part 1
  • Part 2
  • Part 3
  • Part 4
  • Part 5
  • Part 6
  • Part 7
• Raymond’s series of posts on iterator blocks
  • Part 1
  • Part 2
  • Part 3

Share:

Background: As per this post, I’m making a simple upload-my-videos-to-YouTube app.  Step 1 was figuring out how we were going to get the info out of FlipShare, which we did in this post.  Step 2 is figuring out how we’re going to actually upload to YouTube, which is this post :)

Investigating available options

Just a couple of days before it was announced on their blog, I had started searching for available API’s (ideally already in .NET) and ran across their YouTube SDK and the larger GData SDK on their project download page.  Awesome!  Their wiki also described a sample app that would be perfect for me to learn from – YouTubeUploader!

One thing I noticed, though, was looking around the filesystem I didn’t see the source for YouTubeUploader.  I could certainly run reflector on it (and did), and that’s certainly a lot better than nothing, but it was odd that all the other samples had source but not this one (the later announcement would explicitly say you had to get the source via subversion).

So, I go to the Source tab to try and figure out where it might be in the tree, but at the time there was only 1 hit in an unrelated unit test. I noticed the UI had a type in it (uplads instead of uploads) so I searched for that string, and no hits. I filed a bug that I couldn’t find the source which Frank from the GData team closed after noting the location in the source tree.  Checking again as I write this, whatever process indexes the source tree has now picked up the source and both searches return parts of the project :)

Trying to use YouTubeUploader

At that point in time, I actually had 5 videos left that hadn’t successfully uploaded with FlipShare, so I tried to use the YouTubeUploader app to get them uploaded.  Its interface to the user is a CSV file you need to create to define title/tags/category/path/etc.  No biggie – I open Excel and make it, then save as csv (I’m too lazy to make the CSV by hand, worrying about quoting strings with commas and the like).

However, when I actually run it, the uploads all fail (doh!) with “400 Bad Request” (that’s what shows up in the UI).  I figured I’ve just got something misconfigured, so I try a bunch of different things in the UI, in a YouTubeUploader.exe.config, etc., but no such luck.

At this point I think about just ditching it here rather than go down the investigatory rabbit hole, but since it’s from the GData team and it’s the sample I’d really like to start with, I press on.

Investigation

So, time to dig in to figure out what’s going on.  I run my current go-to tool for HTTP debugging, Fiddler, and then restart the app and have it try again.  As I should have expected, it does the calls over https and Fiddler in the middle is breaking all the calls since it’s not a trusted CA.  I hadn’t actually added Fiddler as a trusted root CA before, but there’s nice simple instructions to do so.

With that working, I look at the request and response of a failed call.  You can see them in the bug I filed, but the problem didn’t jump out at me at first.  I googled the error message (“Content is not allowed in prolog”) and much like you’d expect, it comes from having stuff show up before the xml prolog (“<?xml … ?>”), breaking the xml parsing (from the hits, it appears to be in Java XML parsing, not that it matters).  So I look back at the request again and I didn’t see anything before the prolog.

However, Fiddler has a ton of different ways of showing the request and response, including a handy-dandy hex view.  That show that there are indeed 3 bytes between our HTTP-spec 2 newlines (separating headers from body) and the xml prolog itself.  The bytes are 0xEF, 0xBB, and 0xBF.  Those bytes seem oddly familiar, and Google reminds me why – it’s the UTF-8 Byte Order Mark.  Ah, yes, it all starts to make sense.

Who’s to blame?

So, more googling and I run across the post that confirms what’s going on with XmlTextWriter along with a fix (Thanks Rick!).

Looking at YouTubeUploader’s source, it’s just using the ResumableUploader class in Google.GData.Client.dll, so it seems clear the bug isn’t YouTubeUploader’s fault.  Since it presumably was working for others, I was wondering if maybe it was something in the BCL (having 4.0 RC on the same box, although it should side-by-side fine and I had checked that YouTubeUploader was still running under 2.0/3.5. 

Stepping through in the debugger, I see the offender – AtomBase.SaveToXml(Stream) does exactly what Rick’s post said – creating the XmlTextWriter with Encoding.UTF8, which writes the BOM. 

Confirming the problem

At the time I was having problems rebuilding things (long story with no value – PEBKAC :) so I wanted to verify that’s the problem by modifying the requests.  I knew Fiddler had this capability but I had never done it before.  Looking through the cookbook samples, though, it seemed pretty straightforward. 

The biggest hurdle ended up being how to get those 3 bytes into a string, although for no good reason.  I knew about GetString off of Encoding but for some bizarre reason I thought that given the UTF8 BOM, it would just discard that (since I thought of it as ‘reading’ the UTF8 bytes) and leave me with an empty string.  Of course, once I finally tried it, it worked just fine and worked fine.  Figures. :)

The BOM isn’t going to change, but rather than put the bytes in directly to the rule, I referenced GetPreamble, which resulted in this simple addition to the OnBeforeRequest handler:

	var utf8: System.Text.Encoding = System.Text.Encoding.UTF8;
	var bom: String = utf8.GetString(utf8.GetPreamble());
	oSession.utilReplaceInRequest(bom + "<?xml", "<?xml");

I ran YouTubeUploader again with that rule in place, and sure enough, the uploads start working fine!  Yay!

Trying the fix

Once I wake up and finally realize that I don’t need to rebuild the full YouTubeUploader app, but instead just the Google.GData.Client.dll (hey, it was late :).  I make the one-line change (svn patch attached to bug), rebuild the dll, then drop it in to Google YouTube SDK for .NET\Samples and try YouTubeUploader again.  Sure enough, it worked fine (and faster, since it didn’t have to go through Fiddler’ and its request rewriting :)

Where are we?

So, while we (well, I) have spent an unfortunate amount of time yak shaving, we have a working sample for how to upload to YouTube.  The sample is pretty incestuous between logic and UI (which is fine, it’s a sample :), but it’s working code, uses the resumable uploader API, already supports multiple simultaneous uploads, and already supports configurable numbers of retries.  It’s already doing the hard parts, so it makes my life much easier writing my little uploader object model and apps. :)

Share:

WARNING: this shows an implementation detail of FlipShare 5.0, NOT a public interface – it could break with upcoming versions

Goal – get the info directly from flipshare.db.

As has been noted in many places, FlipShare uses a SQLite database for storing its data.  When you add titles (or, well, change them from “Untitled” to something else), it doesn’t modify the names of your video files on disk, it just updates a row in the database.  Same for moving to a new logical folder within FlipShare (and lots other data changes, of course). It’s a great decision on their part, IMHO, even if it makes interacting with your videos in the filesystem a bit more painful :)

If you’re trying to interact with the actual videos on disk, you’ll notice there’s no description in them at all – look under Videos\FlipShare Data\Videos and you see VIDnnnnn.mp4 files (where nnnnn appears to be a normal monotonically increasing integer, at least at first glance unrelated to any of the PK id’s in the database).

Since we want to upload our videos with the (arguably, meta)data that’s shown in the FlipShare UI (logical folders, titles, etc), we need to access that flipshare.db sitting in your Videos\FlipShare Data folder.  We could use FlipShare’s export feature, but that would mean taking more disk space than required and we’d have to re-export if we wanted to ‘sync out’ changes to titles, and AFAICT it wouldn’t reflect the logical folders (at least not automatically), just the titles.

Schema investigation

The closest I found to an existing schema explanation was this post which admittedly does give you the 3 tables that hold the pieces of data you need – the title, the video location on disk, and (kinda optional if you just want to upload the video by title) the logical folder it’s in.

The problem for me was figuring out how to join these together.  SQLite supports foreign keys (well, at least 3.x does, not sure if that’s new though), but flipshare.db doesn’t use them, so I started to look at the data in my db’s tables to see if I could just notice it from the data.  The 2 tables I cared the most about were MediaElement and MediaElementSource since they hold the title and video file path, respectively.

• I looked at the data of each to try and find the join condition, but nothing matched up.
  • so, it’ll be more than a simple 2-table join
• At this point I wanted to look for a simple mapping table (so, a 3-table join)
  • I picked a particular video in MediaElement based on title,
  • then looked it up in the FlipShare UI to get its length
  • then sorted the VID*.mp4 files by length and looked to find the right video (the one that matches what’s show in the FlipShare UI)
  • then found the MediaElementSource row that matched up
  • Now I had the PK values for the 2 tables I cared about for a single video.
  • Now I looked for data in the database (using .output and .dump in sqlite3.exe) that referenced those 2 values (in either order, of course), hoping there would be a single mapping table involved.  No such luck.
• At that point, knowing it must be a join of more than 3 tables, I gave up on trying to inspect my way to the join conditions.
• I did findstr /sim in the Program Files\Flip Video\FlipShare to find which places referenced those tables, which had hits for Core.dll and FlipShare.dll
• Since Core.dll seemed the more likely place for the DAL I then ran strings.exe on the file, dumped it out to a text file, then started searching for the table names in the output.
• I found a query that joined MediaElementSourceGraph (which I had already noticed during the manual schema inspection was linked to MediaElementSource by its mediaSourceId – nice, friendly FK name, even if not defined strictly as a foreign key) and MediaElementRendition (which I had similarly noticed was linked to MediaElement via mediaElementId, also clearly an FK name).
• So, the final 4-table join would link MediaElementSource to MediaElementSourceGraph to MediaElementRendition to MediaElement
• In retrospect, the schema makes sense, of course – I just couldn’t see the forest for the trees :)
• As a bonus, we could add in the logical folder location for the video, which was much easier to figure out since UserFolderMediaElements is a simple mapping of videos (MediaElement rows) to logical folders (UserFolders rows) as you could tell by its name, so it was just adding those 2 more tables (6 total) to get all the info we’re looking for.

Building the query

Using LinqPad 2.x with the IQ driver (so we can query SQLite) we can use linq’s support for joining (even without foreign keys in place).  It defaults to a couple of common conventions I’ve seen in other OR mappers (linq-to-X and otherwise), making the table names plural (but not the row entity singular if the table is already plural, oddly enough IMHO) and init-cap’ing the property names.  You could certainly turn those off, the query would just need to be slightly different.

I hit one gotcha while constructing the query and adding in the 2 folder-related tables, though:

It took me awhile to figure out because I incorrectly took the error message to mean it couldn’t figure out the type of ‘folderElem’ (and adding the explicit type didn’t change the error, of course :), but I eventually figured out it was because of a type mismatch between the members involved in the join condition (elem.Id equals folderElem.MediaElementId).  The UserFolderMediaElements table, oddly enough, doesn’t store the mediaElementId column as an int or similar numeric column – instead, it’s a string.  Not sure why that is (perhaps hysterical raisins), but it is.  Fixing it was simple enough, though – just ToString the elem.Id so it’s comparing 2 strings.  (You can’t use Convert.ToInt32 on the folderElem.MediaElementId since that method isn’t supported by the driver, at least not yet :)

One optional where clause (commented out below) is checking the folder’s ParentId – for the FlipShare logical folders (under their ‘Computer’ node), the parent id is 8 (at least in my database :) – i don’t see this in another table, so I’m guessing it’s a defined constant in the code (looks like folder id’s under 1000 are likely that way).  I don’t need this at the moment since I don’t have any videos in other areas (like the ‘Flip Channels’ node), but if you do, you might want that filter.

LINQ Query to get the video info

from source in this.MediaElementSources
    join graph in this.MediaElementSourceGraphs
        on source.Id equals graph.MediaSourceId
    join rend in this.MediaElementRenditions
        on graph.RenditionId equals rend.Id
    join elem in this.MediaElements
        on rend.MediaElementId equals elem.Id
    join folderElem in this.UserFolderMediaElements
        on elem.Id.ToString() equals folderElem.MediaElementId
    join folder in this.UserFolders
        on folderElem.Id equals folder.Id
//where folder.ParentId == 8
select new
{
    elem.Name,
    elem.PreviewImagePath,
    source.Uri,
    rend.VideoFormat,
    folder.FolderName,
}

NOTE: the VideoFormat isn’t really all that useful to have in the output, I had just included it because I was curious, but feel free to kill it if you don’t need it.  It’s “free” in that you have to join on MediaElementRendition anyway, of course :)

SQL Query to get the video info

Since this is LinqPad, I just need to click on the SQL button to get the query that the LINQ query was transformed into, which I include here since it’s more likely to be applicable/usable for people running across this post :)

Admittedly, the generated SQL doesn’t have the nicest table alias names, but that’s pretty easy to search-and-replace (or just remove them) if someone wants to :)

SELECT t0.[name], t0.[PreviewImagePath], t1.[uri], t2.[videoFormat], t3.[folderName]
FROM [MediaElementSource] AS t1
INNER JOIN [MediaElementSourceGraph] AS t4
  ON (t1.[id] = t4.[mediaSourceId])
INNER JOIN [MediaElementRendition] AS t2
  ON (t4.[renditionId] = t2.[id])
INNER JOIN [MediaElement] AS t0
  ON (t2.[mediaElementId] = t0.[id])
INNER JOIN [UserFolderMediaElements] AS t5
  ON (t0.[id] = t5.[mediaElementId])
INNER JOIN [UserFolders] AS t3
  ON (t5.[id] = t3.[id])

Obligatory Screenshot

Here’s the results from LinqPad, showing it’s the desired output :)

And, to finish off the post, a repeat of the warning we saw at the beginning, just in case someone skipped down to get the queries :)

WARNING: this shows an implementation detail of FlipShare 5.0, NOT a public interface – it could break with upcoming versions

Share:

I got asked what my impressions are of the Flip UltraHD I’m testing out.  The hardware itself is fine, although a little mediocre in 2010 since the bar’s constantly going up (720p instead of 1080p, no image stabilization, but pretty good in low-light, and the “candy bar” form factor is more usable than I expected), but FlipShare, the software that comes with it, is the linchpin for the overall user experience that is really the main goal on selling it (something akin to Apple product experiences, IMHO).

The box came with version 4.5 of FlipShare which had a good chunk of things I found annoying, but after the upgrade to their 5.0 version (5.0.5.52727 at the moment), the majority of those went away – it’s a nice, solid, very usable interface.  Good WAF, too.

It can upload to multiple services (facebook, myspace, youtube) – I picked youtube as a destination mainly for popularity – since it’s extended family that will be viewing these, I wanted a site they’d most likely already be somewhat familiar with.  I know a good chunk of them still aren’t on facebook, and most of them probably never even heard of myspace :)

The upload behavior is enough such that I’ve started my own app that’ll upload the videos (to be covered more in posts to come), especially after I threw in the various “bonus” bits and realized it’d be pretty unlikely that all this gets taken care of by Cisco+LinkSys+Flip in a timeframe I’d be ok with :)  It also gives me a chance to take advantage of some of the new/better out-of-browser capabilities in Silverlight 4, although there will be cmdline and desktop-app versions as well :)

Problems with FlipShare’s uploading

• I always upload to YouTube – let me say "always this", or at least default the radio button to what was last selected so I don’t have to keep clicking it (similar to how the visibility defaults to public instead of private later in the "wizard")

• When uploading to YouTube, there’s 2 phases involved:
  1. For a LONG time, FlipShare isn’t actually uploading anything and is just hogging CPU. Is it transcoding or something? It doesn’t (or at least, shouldn’t) need to (already H.264 MP4 coming off device).  I can upload the files directly with no transcoding using other youtube uploaders.
  2. After that it actually does the real upload – I can tell when it starts because the CPU fan in my laptop quiets down considerably :)

• No X retries (would be very helpful when the laptop sleeps+resumes, since it’ll definitely fail during that cycle and will need to retry)
• Related, apparently no resumable uploads?
• If an upload job fails, all the videos in that upload job appear to fail
• When a job fails, especially if you had multiple upload jobs going in parallel, you can’t tell which videos didn’t make it without inspecting what’s on youtube.
• No ability to view the details of the actively running upload jobs, like:
  • Rate of transfer
  • List of files and their individual progress (completed/transferring/pending/etc)
  • Estimated time of completion (per file and overall), both as X hours and potentially as a time of day for users that don’t want to do the math themselves :)

• No notifications of progress/finishing
  • Toast on desktop (like Outlook)
  • Emails
  • Bonus:
    • Facebook, Twitter, whatever notifications so family and friends find out about new videos (do per batch, not per video!)
    • NOTE: YouTube has this natively via AutoShare as a workaround for now, but I’d rather have it in the FlipShare UI so I can more easily customize the notifications (recipients, added text, etc) beforehand
      • http://www.google.com/support/youtube/bin/answer.py?hl=en&answer=157215

• Bonus: allow "syncing" of videos up to YouTube such that it checks whether a file is already up there (name, length, hash, whatever) and skips it if so.
  • This lets people just worry about maintaining a local copy and let it push up automatically, something akin to SyncToy
  • Target is eventually running on WHS where you get your videos onto your \\server\videos share and a WHS plugin syncs them up to YouTube for you

• Bonus: allow creation of YouTube playlists based on folder structure within FlipShare
  • Similarly needs to be kept in sync – if you move videos around to different folders, it should modify the playlists

Share:

Remember that previous post about List<string>.Add?  Well, one of the uses of the messages was to get into an email.

Unfortunately, there’s not an Attachment ctor that takes the contents of a file – it’s all around passing in a filename or a stream.  Since I don’t want to have to write this out to disk, stream’s the way to go.

NOTE: you would be forgiven if you read the example code in the string-param ctor and thought it actually *did* take content as the param.  I’m not sure the author of the example knew that the param was a filename, to be honest (although it’s possible the example was written during the 2.0 cycle when that ctor really did take content).  The example’s been busted for awhile. :)  Even the second comment here is wrong – it’s not in the body, it’s an attachment! (API perspective, ignore mime encoding behavior :)

// Attach the message string to this e-mail message.
Attachment data = new Attachment(textMessage);
// Send textMessage as part of the e-mail body.
message.Attachments.Add(data);

Creating the Content

Now, on to getting the messages into file content.  Sure you could foreach and WriteLine something (for instance, new StreamWriter(new MemoryStream) then Position = 0 the stream afterwards for the ctor or whatever), but I like more declarative-ish constructs and like to actually be able to easily see (my infinitives are very flexible – they can do a split!) the file content in the debugger, so:

var testingContent = String.Concat(testingMessages
    .Select(s => s + Environment.NewLine) // append a newline to each
    .ToArray()); // make array so Concat can take it

Admittedly that last line is annoying, but it’s only there because this is currently built against 3.5 – in 4.0 String.Concat (and String.Join) thankfully added IEnumerable overloads

Then it’s a matter of constructing a stream (I’d use UTF8 or Unicode normally, but wanted ASCII for this)

new MemoryStream(Encoding.ASCII.GetBytes(testingContent))

then passing it to the ctor (and then adding that to the message)

message.Attachments.Add(new Attachment(testingMemoryStream, "text/plain")
    {
        Name = "testingMessages.txt",
    });

Attachments and Disposed Streams

One minor gotcha you may run across – the Attachment (smartly) doesn’t pull anything out of the stream at ctor time – the data will be fetched when you actually send the message (since at that point it needs the data to actually write the outgoing email), so while you should do a using() or similar to dispose the MemoryStream you create, you need to make sure that doesn’t happen until after the email is sent – otherwise you’ll get an exception when it tries to read the disposed stream :) 

You may be tempted to using (new MemoryStream) { message.Attachments.Add(…); } smtpClient.Send(message); but don’t do it! :)

Memory Usage?

Since someone will likely point it out – I know the memory behavior of this isn’t great – we end up with the same content in 3 places: 1) the List<string> 2) the file content 3) the memory stream’s byte buffer.  You could certainly have created the MemoryStream, wrapped it in a StreamWriter, then written the messages to it instead of the List<string>, reset the position before passing to the ctor, and been down to 1 copy of the contents in memory instead of 3.

Mitigating factors are: 1) This code path isn’t used in production and 2) we’re not keeping the copies around very long – the first 2 copies are free to be GC’d after the Add of the attachment since they’re local vars that go out of scope.

Share: