I thought it would be nice to share some of the technical books that I have learnt most from, and still love to look through or use as reference to this day.   These are in no particular order and cover a variety of technical subjects, and some are far from recent.  If you are interested in any of these areas and have not read these books, I cannot recommend them highly enough!  I have many, many many books and have read countless over the years, many of them are awesome but are not making this list as it would just be too long.  I also have lots of books that are simply terrible (and I still have most of those too…!)

The Black Art of Video Game Console Design,  Andre Lamothé

Well, this book just has it all really.  I already knew basic analogue and digital electronics when I first bought this book (learnt from a whole host of other books), but it taught me most of what I know today about computer design.  Starting from basic electrical theory (physics) and analogue / discrete electronics and circuit analysis, it quickly moves into semiconductor theory and digital logic.  From there, the first half of the book is building on previous topics, warming up for the second half.  Basic logic gates, various forms of adders, ALU design, shift registers, multiplexers, buffers, etc (and a whole bunch of theory).  The second half of the book concentrates on embedded system design and computer architecture.  This will literally show you how to design your own processors, a multitude of data bus / pipelining / cache / multi-core architectures and how to glue everything together with various different memory addressing strategies and so forth.   Finally the book goes into details on how to take your new computer architecture and build video / sound / input / more memory /  extensions points and create a video game console. Be prepared for a hard crash-course on PAL / NSTSC.

From this book I actually built several variations of my own computer that had graphics and everything. Proudest moment, getting a stable green bar to appear on my PAL monitor. can’t recommend this book enough.

link

Computer Systems – A Progammers Perspective, Randal E. Bryant & David O’Hallaron

Wow, another awesome book that I leant so much from, so many years ago.  This is not your normal programming book, this is a bit of an odd hybrid that sits somewhere between computer engineering and programming.  Essentially this book is more about how your computers actually work, but flavoured from a programmer’s perspective.  All the code in this book is presented in C or Assembly,which is suitable is the concepts it covers are very low level.  From the basics of how information is represented in a computer (and the various different versions of it) along with internal data type representation and arithmetic, it moves quickly onto how your programs are really represented at a machine level.  It goes into fine detail on exactly how your programs work, then moving onto processor architectures (and how that affects your programs) and optimizers within compilers and so on. Finally it covers in-depth discussions of memory hierarchies, virtual memory, networking, etc. 

I still enjoy leafing trough and reading bits of this book, it is rare a book goes into this much detail and is as well written.  Also, this book has Exercises in it! Woo :)

nb. There is a newer edition of this book from 2010! I don’t know what differences it has.

link

Amiga Game Maker’s Manual, Stephen Hill

A long time ago, in a galaxy far away, I was about 14 working at Maplin Electronics on my Work Experience from school.  One of the first things they had me do (the bastards) is finish cleaning out their old warehouse.  Amongst the dust and the debris I happened across this book.  Now, given my Father owned a small Amiga company selling public domain software, I was already surrounded in Amigas and had both Blitz and Amos basic, and had been attempting with some moderate success to write computer games for many years.  Afterall, that’s the main reason that most of us got into it right?  Now, whilst this book is not particularly groundbreaking in many ways, it had a unique charm and style.  It was categorised into some chapters for general programming, and then a chapter devoted to the various game genres, from arcade shoot-em-up through RPG and even full on simulator games. At the end of each chapter it would have a list of other ideas or themes for games related to that genre.

Therefore is was from this book I leant about how to use trig to make cool attack patterns in shoot-em-ups, how to actually design a sprite animation system properly, an so on.  I learnt tons from this book (and some really bad stuff!) but more importantly it really helped spark my imagination more into what was possible and how it could be achieved.

Proud to say I still have this book (somewhere!) :)

link

C++ How to Program, Paul Dietel

Well, this is a tough one to write about.  I have over the years bought numerous Dietel books and they are largely terrible in my opinion (although some make great doorstops).  This is the fist one I got and was one of the first C++ books I owned.  The one I have is a later edition, which I understand now is a lot better than the originals.  At any rate, this is a fairly thorough coverage of the C++ language, OO design and data structure implementations.  I had better books since, and some worse ones before this - but it earns a place in my top books as it is was the first one that had the language really make sense to me.  It has some horrible UML stuff in it, overblown OO design guidelines, some waffling about not much in places, but overall nitpicking aside this book was actually pretty awesome in my opinion and paved the road to C++ for me.

Also, being a university text book, it has EXCERCISES in it.  I love those :)

link

Principles of Computer Hardware, Alan Clements

This is similar in some ways to the first book in this list, although only dealing with digital logic and a lot more on the assembly programming side of things. Actually it’s kind of like a mix between the video game console book and the computer systems one. Much like Andre’s book, this starts off with the basic logic stuff, gates, boolean algebra, karnaugh maps and so on,quickly progressing into very logic-gate-heavy diagrams of pretty much all the systems in your computer.  Once again this goes into fine detail on CPU architecture, pipelining and so on, but with more an emphasis on the top level rather than the electronics.

I think with the video game console book, this one, and the computer systems one you should be very knowledgeable on all things software and hardware!

link

Tricks of the 3D Game Programming Gurus, Andre Lamothé

Another one of Andrés books (can you tell, I am a fan).  If you are anything like me, you will not be happy until you really understand how stuff actually works.  I am happy with some black boxes for a while (a skill I struggled with until later in life!) but ultimately I am going to want to know how those black boxes really work.  I was probably about 18 when I decided I want to learn how to program 3D video games.  There are tons of books out there on how to use DirectX or OpenGL, and I had learnt a bunch of stuff from various tutorials on the web, but none of them really showed me how stuff works – until I bought this book :)

Whilst this book does show you how to use DirectX, it goes much further than that, delving into most rasterization theory and practice, it will have you build a software renderer from the ground up.  Most of the black boxes you use in 3D programming are explained in detail here, including a very friendly primer on 3d maths which was essential for me as I’d done basically no maths since year 10 in high school, other than some bits of trig in games I had written (shoot-em-ups, raycasters, etc).  This book starts with Win32 programming and GDI, moving onto DirectDraw7 and then Direct3D later.  Andrés typical style here is to show you how to do everything the real hard way, then show you a Direct X function that does it for you.   This book is not for the feint of heart (or lack of bicep muscles, it’s not a small book), but a rare gem in a sea of sub-par 3d texts.  I wrote my own 3D game engine from using this book (which I lost in the great hard-drive crash of 2005).

Note: I have the second edition of this book as well, both of which are well out of date now, but both can and will still teach the real fundamentals that most books go nowhere near.

link to 2nd ed

The Algorithm Design Manual, Steven S. Skiena (2nd Ed)

This book gets some stick as not being that great, but personally (and for a couple of friends) I found this book quite fascinating and very educational.  As is implied, this is a book on not only many different algorithms over various data structures, but also on the analysis and design procedure of designing an algorithm to fit a given problem. I especially like the “War Stories” sections where the author explains in detail how he used said algorithms to solve real-world problems. To give a flavour of topics covered; algorithm analysis (big oh notation) data structures, sorting / searching,  graph traversal, weighted graph algorithms, combinatorial search, dynamic programming, approximation algorithms.

This book also has exercises :)  It is a great one to have on the shelf as a reference book, and also an entertaining read in many areas,

link

Genetic Programming: On the Programming of Computers by Means of Natural Selection v. 1, John R Koza

If you don’t already know, Koza is the guy that pioneered genetic programming, and this was the first book as a result of it.  After being fascinated a long time with genetic programming and genetic algorithms, I finally bought this book.  Mind = blown.   Learning LISP aside, this extremely detailed book on genetics lead me on my adventures into this space and was quite mind expanding in terms of how I might create Skynet in the future (not to mention some insights into natural selection and real genetics).  AI is something I don’t really have time for any more, but I still have a bunch of experiments derived from this book (some of which were integrated with my various robots), and also in turn led to me examine other AI techniques in a lot more detail.

link

Expert F# 2.0 Don Syme , Adam Granicz , Antonio Cisternino

This book was the one that really got me into functional programming.  I have read it several times, and some areas of it still blow my mind.  This book is not perfect, in my opinion it glosses over some important advanced stuff (like Quotations) and instead chooses to put a bunch of other content in which in my mind is not so much to do with the language – and this is a problem being the definitive book for F#. It also has no exercises which it would certainly benefit greatly from.  Bitching aside however, this is up there with one of the books I have learnt most from, insomuch as it largely turned upside down a lot of ideas I had about programming computers and displayed a whole new side to everything in computer science and software engineering, which was a key turning point in various areas for me. 

link (to 3rd ed)

Game Programming Gems and AI Game Programming Wisdom series

I have mostly all the books from these series.  These books are not cheap, and come in hardcover only, but they are lovely things to own :)  What I really love about these books is they are current (well, they were, haha).  The format of the books is a series of articles written by various engineers working for big 3d game studios around the time of publication.  The articles are heavily edited in a very professional manner, and cover an extremely large amount of topics in game programming.  These books are very cool because they show you what the new cutting edge techniques being used at the time were, which is something most books are not able to do.  The disjoined nature of the articles, being roughly organised into categories is a big plus as you can study and article about a particular concept and play around with the sample code without having to get wrapped up in the whole book to do so.  A lot of the techniques in these books helped me think about many areas of software differently, and I adapted several concepts from these books into various projects of my own.

Plus you could always look forward to the next book in the series, and was a good thing for my parents to get me for birthdays and Christmases :)

Conclusion

Whew, I am going to leave it there with 10 entries. I have many other books which are equally deserving to be on this list, especially around the areas of programming language concepts, compilers, game programming and hardware / electronics books.  I think this is fairly well balanced on the my interests throughout life so far though.

What are some of your favourite tech books?  I would love to know.  Write an article and share, or just let me know by tweeting me @pezi_pink !

The SQL provider now supports basic transactional CRUD functionality and an ODBC provider. A new nuget package is up for you to grab here. As always, you can download and build from source here.

The nuget package is still pre-release. You can find it in Visual Studio by toggling the search filter to include pre-release packages. I'm sure Xamarin has a similar feature.  Once this work has been tested well enough,  I will likely upgrade the SQL Provider to a proper release.

Experimental ODBC Support

We now have support for general ODBC connectivity in the SQL provider.  This provides us with awesome power to connect to almost anything, including Excel spreadsheets.  However, because each driver can vary substantially, not all drivers may work, and others may have reduced functionality. ODBC is provided in the .NET core libraries, although you will of course need the appropriate ODBC driver installed on your machine for it to function.

ODBC has Where, Select and Join  support.  If  the source in question provides primary key information, you will also get Individuals. Although explicit joins are supported, foreign key constraint information and navigation are not yet available. The new CRUD operations are also supported, where appropriate.  We will see how this works from ODBC later in the post.

[<Literal>] 
let excelCs = @"Driver={Microsoft Excel Driver (*.xls)};DriverId=790;Dbq=I:\test.xls;DefaultDir=I:\;" 
type xl = SqlDataProvider<excelCs, Common.DatabaseProviderTypes.ODBC>

You can use the new features with the following providers :

• SQL Server
• SQLite
• PostgreSQL
• MySQL
• Oracle
• ODBC (limited)

The following examples will use SQLite, but the mechanics are identical for all the providers.

CRUD Operations

IMPORTANT IMFORMATION ON RESTRICTIONS!

The CRUD operations will only work for tables that have a well-defined, non composite primary key.  Identity keys are fully supported.

The data context is the core object that enables CRUD. It is important to understand that each data context you create from the type provider will track all entities that are modified from it.  In a sense, the data context “owns” all of the entities returned by queries or that you have created.  You can create more than one data context that have different connection strings at runtime.  You can pass a different connection string in the GetDataContext method.  This is an important concept as it allows you to connect to multiple instances of the same database, and even replicate data between them fairly easily.

type sql = SqlDataProvider< ConnectionString, Common.DatabaseProviderTypes.SQLITE, ResolutionPath > 
let ctx = sql.GetDataContext() 
let ctx2 = sql.GetDataContext("some other connection string")

whenever you select entire entities from a data context, be that by a query expression or an Individual, the data context involved will track the entity.  You can make changes to the fields by setting the relevant properties.  You do not need to do anything else, as the data context handles everything for you. 

let hardy = ctx.``[main].[Customers]``.Individuals.``As ContactName``.``AROUT, Thomas Hardy`` 
hardy.ContactName <- "Pezi the Pink Squirrel"

ctx.SubmitUpdates()

Note the call to SubmitUpdates() on the data context.  This will take all pending changes to all entities tracked by the data context, and execute them in a transaction.  An interesting property of the above code is that after Thomas Hardy has had his name changed, the first line will no longer compile! Go F#!

Similarly, you can delete an entity by calling the Delete() method on it.  This action will put the entity into a pending delete state.

hardy.Delete()

ctx.SubmitUpdates()

After the deletion is committed, the data context will remove the primary key from the entity instance.

Creation is slightly different.  You will find various Create methods on the IQueryable Set types that represent the tables.  Up to 3 different overloads of this method will be available.  The first takes no parameters and will return a new entity which are you expected to populate with at least the required fields.  A second version will accept the required fields as parameters – this is only available if there are any columns marked as NOT NULL.  The final version will create an entity from a (string * obj) sequence – it is potentially unsafe but very handy for copying entities or creating them from some stream of data, if you know the attribute / column names are correct.

A note on primary keys – presently the SQL provider does not detect identity columns, although it does deal with them on the insert appropriately.  You will never see a primary key field as a parameter to the Create method.  If your primary key is an identity column, you can simply not include it.  Upon insert, the provider will automatically update the instance of the entity with the ID it has been assigned.  If on the other hand your key is not an identity column, you will be expected to assign it an appropriate value before attempting to submit the changes to the database.

// create an employee 
let pez = ctx.``[main].[Employees]``.Create("Pezi","Squirrel") 
ctx.SubmitUpdates()

printfn "Pezi's new employee id is %i" pez.EmployeeID

//update something 
pez.Address <- "the forest" 
ctx.SubmitUpdates()

// delete 
pez.Delete() 
ctx.SubmitUpdates()

Now let’s see how we can effortlessly combine different SQLProviders together in order to perform "extract, transform, load" type processes.

type sql = SqlDataProvider<northwindCs, UseOptionTypes = true> 
type xl = SqlDataProvider<excelCs, Common.DatabaseProviderTypes.ODBC>

let northwind = sql.GetDataContext() 
let spreadsheet = xl.GetDataContext()

// load our tasty new employees from our lovely spreadsheet 
let newEmployees = 
    spreadsheet.``[].[Sheet1$]`` 
    |> Seq.map(fun e -> 
        let emp = northwind.``[dbo].[Employees]``.Create() 
        emp.FirstName <- e.FirstName 
        emp.LastName <- e.LastName 
        emp) 
    |> Seq.toList

// save those puppies away 
northwind.SubmitUpdates()

In this sample we use the ODBC type provider to gain instant typed access to an Excel spreadsheet that contains some data about new employees.  the SQL Server type provider is also used to connect to a Northwind database instance.

We then very simply pull all the rows from the spreadsheet, map them to an Employee database record, and save them away.  It doesn’t get much easier than this!

Or does it?  Actually, in this case we know up front that the spreadsheet contains identical field names to that of the target database.  In this case we can use the overload of Create that accepts a sequence of data – this method is unsafe, but if you know that the names match up it means you don’t have to manually map any field names :)

let newEmployees = 
    spreadsheet.``[].[Sheet1$]`` 
    |> Seq.map(fun e -> northwind.``[dbo].[Employees]``.Create(e.ColumnValues)) 
    |> Seq.toList

I think we can agree it really does not get much easier than that!  You can also use this technique to easily copy data between different data contexts, as long as you watch out for primary and foreign keys where applicable.

Data Binding

The SQLProvider, with a fair amount of trickery, supports two-way data binding over its entities.  This works together very well with the CRUD operations. 

open System.Windows.Forms 
open System.ComponentModel

let data = BindingList(Seq.toArray ctx.``[main].[Customers]``) 
let form = new Form(Text="Edit Customers") 
let dg = new DataGridView(Dock = DockStyle.Fill,DataSource=data) 
form.Controls.Add dg 
form.Show()

This is all the code you need to create a fully editable data grid.  A call to SubmitUpdates() afterwards will push all the changes to the database.

Other Bits

The data context now has a few new methods on it that you can use.

1. ClearPendingChanges()  :  Ronseal.  Remove any tracked entities that have changed. Use this with caution, as subsequent changes to the entities will be tracked, but the previous changes will have been lost.
2. GetPendingChanges() : This function will return a list of the entities the data context has tracked.  This is useful in a variety of situations, and it also means you do not have to bind to created or updated entities in order to not “lose” them in your program.