Mapping Custom Queries in Java Spring

The scenario is that you have to get information from two different tables that are related. You can’t just query results from a single table which would fit neatly in an existing model. You’ll want something like this. I’ll explain what I can after the code.

public interface FooRepo extends JpaRepository<Foo> {  
  // Why no mutliline string literals, Java?!!! lololol - salty JS dev
  @Query(value="SELECT "
              + "foo_id AS fooId, "
              + "bar_id AS barId "
              + "FROM tbl_foo "
              + "LEFT JOIN tbl_bar b "
              +" ON b.foo_for_key = foo_id",
  List<FooBarDTO> findByEmailAddress();
public interface FooBarDTO {  String getFooId();  String getBarId();}

Notice the AS usage in the field selection matches the getter naming convention. Like if you had private fields in the DTO named fooId/barId and Lambok created your getters and setters with @Data or @Value or whatever.

I have no clue how we get an instance of a class from this, but the mapping works and you get your data. To be clear, you can not declare and implement this interface. Just do this alone. Yes, it makes no sense. Yes, that is not how interfaces work. Yes, this is how Spring does it anyway. Maybe this is normal for Java, but I found it to be a really weird usage of interfaces.

If you are feeling more brave, I was able to find the relevant documentation hiding here. Be careful trying to click on any section links, many don’t go to the relevant subsections and you can easily get lost trying to find anything.

Hello from elm-pages! 🚀

In my apparently never ending quest for more complicated solutions, I’ve now decided to move my blog over to elm-pages.

It’s a static site generator like Gatsby, and it does the similar sorts of optimizations that Gatsby does. Only it is based on Elm. I have growing love affair with Elm, and since my day job already includes an enormous amount of React work, I figured I’d use the site revamp and an excuse to migrate.

I originally move the site over to Gatsby for similar reasons when I was first starting out with React. So this is an opportunity to utilize elm more and demo some of the things I’m doing with Elm, as well convince myself that Elm can handle the use cases I would need React for.

So what was wrong with Gatsby or React? Well, nothing wrong per se. Both are fine status quo solutions. I’m proud of the work I can do and have done with both of these solutions.

But I’m currently on a kick to evaluate tooling and technologies that will make my development easier and more robust. A big part of that is being unsatisfied with Javascript, particularly for larger projects. I’ve use Typescript and Flow, and linters and Redux and Immutablejs but those solutions are brittle and cumbersome.

Elm is part of that evaluation and thus far been extremely positive. It doesn’t suffer from many of the issues I have with Javascript and it’s looking great so far, but before I go trumpeting how amazing Elm is, and it is amazing, to stakeholders I need to know exactly what can or can not be done efficiently with the technology. As a bonus, the project uses elm-ui and it’s already setup, so I don’t need to fiddle with CSS. elm-ui really deserves it’s own post, but you can check it out here. Be sure to watch the talk about it at elm-conf.

So this is one small step. I’m loving working with Elm and the process of moving over has taken only a couple of hours. And honestly much of that time has been taken up by cleaning up the existing markdown files, and fighting with vim (I’m also learning vim, so yes, I hate myself right now).

We’ll have to see how the deployment goes, but it looks fairly straightforward. If you are reading this sentence, it went reasonably smoothly and I didn’t need to edit this closing statement. Wish me luck!

The Beginner Developer Wonderland

The path to learn to program is an exciting road. It used to be a rather obscure trail. But today there is a wealth of information available online that will guide you along on just about every language and technology under the sun. Nearly any topic that you wish to learn can be taught to you with endless variety from highly skilled teachers and effective materials that make the topics approachable and engaging. And quite often, these teachers are active, professional developers who have hard won experience and tips about the practical aspects of software development. It is truly a golden road for education. A budding developer with enough grit has a very efficient, and effective path to a lucrative career, and the resulting prosperity no matter what their socioeconomic background. One must only take the first step on that journey.

But there is nasty, secret bog hidden along the path that inexperienced developers should be wary of.
I call it the intermediate developer wonderland, where all the tutorials are trivial, where you are infinitely switching between languages and frameworks, and never proceeding to more substantial projects of your own. This is where you take one course or video series about a language, and instead of going on to build more and more substantial projects utilizing your newly found knowledge, instead the poor student opens up another course or video series about some other technology that all the blogs and journals are saying you need to know. The unsuspecting fool doesn’t stand a chance. Even professional developers are prone to industry hype, but they have establish skills to fallback on and put food on the table. The torrent of innovation coming from the industry, and the popular literature of the media leaves the impression that if you aren’t learning some new technology, right now!, that you are falling behind on your professional development.

It’s rough, and it is an easy trap to fall into. The content is so well designed and executed, and makes you feel like you are learning something important when you are really running in place. I could, and have, watched Derek Banas for hours teaching some language or technology that I’ll never used. I’ve spent hours reading and watching talks about Angular, or Aurelia, or Dart, or ClojureScript, or Reason, or Python and the list goes on an on.

Fortunately, Should you find yourself in this position early in your development, and should you find yourself fortunate to happen across this poorly constructed article, there is an easy solution.
The key strategy is to remove choice and thus avoid the resulting analysis paralysis. I am going to outline a plan for every budding developer to follow. it won’t be optimal for everyone, but it’s going to be as effective, and generalized, and simple as I believe is possible.

Here’s the plan, I’ll explain a bit more about the choices later in this article:

  1. Go to
  2. Do all of the lessons in order. (You don’t know enough to know when it’s ok to skip things. When you know enough, you don’t need the program at all.)
  3. Complete all the projects and certs along the way.
  4. If you get stuck at any point, you are going to reach out to the FCC community for assistance. In fact, you should try to be active in the community. You’ll build professional relationships, and it will help with your communication skills, which are extremely important.
  5. You will only graduate from this plan when you have a full time job as a developer or you complete the full program.

You will not deviate from this curriculum except for these reasons.

  • You have a personal project idea you are very passionate about and that you are actively working successfully towards competition. You have ingratiate yourself with an open source project as a regular active contributor and you have people that are actively mentoring you within the project.
  • You have gotten an internship in development and have regular contact with senior developers to guide your progress.
  • You no are no longer interested in become a developer.
  • You will go back to the program if any of these are not the case.

You will most likely be employed as a developer before you complete the program. You will start applying to jobs after you complete the Front End Libraries Cert.

After you are gainfully employed as a developer, then you can start learning other technologies.
Now us you may argue about alternatives to FCC. But it isn’t important. What is important is that Javascript is extremely easy to learn, very high demand, and has low entry level requirements. And the FCC community is huge and active. This is why I chose it, and why other options are only muddying the waters with choice and less optimal consideration for job opportunities.

Yes, you can learn Django or Rails, and there are great free online programs for those, but there are 10 JS jobs for every one of those other stacks, and JS has true multiplatform viability and demand which gives greater long term return on investment and flexibility. JS is simply the optimal choice even considering local job market oddities. Any company that touches the web is going to need JS developers in all but the most dead market.

The most important key is to pick something and run with it.

And then… Build. Build. Build.

Programming is not memorization of facts or syntax. It is not like cramming dates for a history test or latin names for a biology exam. It is a skill that must be developed through practice like a golf swing, painting, writing, or playing an instrument. You have to actually do the work and practice to develop the ability. You are going to write crappy programs. A LOT OF CRAPPY PROGRAMS. Don’t worry about it, you’ll get better on the next project. The key is the volume of work, not the quality. You can and will develop quality as you go along, and have the required context and experience to understand what quality means.

If you are not frequently at least uncomfortable about the thing you are trying to do, you aren’t learning. You are stagnating. Comfort is the enemy, growth comes from discomfort.

And here’s some additional tips.

Write down some affirmations and put them where you will see the daily. I have often put them in my daily schedule, so that every time I get a notification for block of time I’ve set aside for something, I get a message reminding me of my determination and keeping me from self destructive doubts. One of my common ones is “fake it until you make it”. It sounds sappy or trite, but it more effective than you may initially believe.

Also I guess that’s a tip itself as well. Make a schedule. I wasn’t the schedule type. In fact, rigid schedules are very much an affront to my nature. But they work. Explicitly identify some short, medium and long team goals, and put aside time time in your schedule to work towards them every day, week and month. You’ll be amazing at how effective it is making tiny little steps towards your goal. In the case of learning to program. Set aside at least an hour each weekday or several hours each weekend to working your development. You will get results. I promise.

These video have also helped me stay on track if I was getting unfocused, as silly as it sounds.

The Journey To My First Dev Job

March 4, 2014 was when I first joined GitHub. Almost exactly 4 years ago from the day that I am writing this. Today I’ve been employed full time as a remote web developer for two months. I want to talk about my journey to full time development. This is mostly to benefit from those who are just now learning to code. You’ve probably been reading a lot about how to get your first development job. You’ve been building a portfolio and studying and practicing, and maybe you think you are ready. Or maybe you’ve just started and you’ve read about the great salaries and how easy it is to learn to code. There are a lot of success stories floating out there on the interwebs, but I want to tell mine, because most of the stories that you read are…optimistic. Maybe I am a huge outlier, and I’ll get to that, but I don’t think many of these ‘learn to code and get a job in 2, 4, 6 months really tell a representative story.

So, a bit about me. Growing up I’ve had access to a computer pretty much from the time that I could read. This may not sound odd now, but I’m nearing 40. My first computer was a Tandy 1000SX. Home computers were affordable, but they weren’t extremely common. Atleast, in Mississippi they weren’t. It was a family computer, but it was basically mine in practice. I spent many a night typing the magic combination of keys that were given in the many BASIC books we had. Making shapes and colors and sounds. I had a friend who had an uncle that was a programmer that gave him a book on Turbo Pascal. We used it to fashion text adventures together when I would visit. To be clear though, I had no real notion of programming back then, only that I was well acquainted with the idea of it and very basic notions of syntax. Being a programmer wasn’t really even on my radar.

I started in IT officially when I join the Marine Corps. My MOS (job) was basically as a help desk technician / network administration. I dabbled a bit with development while working as a logistics clerk for the company office. (Everyone has two ‘jobs’ in the Marines, your official one, and some duty you may be assigned temporarily.) I automated much of our administrative tasks by building a database and using various office suite software. This software was later moved up to the battalion level and with the help of new Lieutenant, I got my first exposure to Visual Basic. After I left the Corps, I worked a few random jobs, but I also started a blog called MCTPhysics. This was the first time that I really started to learn to program. I wanted to make some collapsible buttons for the categories on my blog. So I hacked together a small script with javascript. I also had to modify templates in what I think was Perl. I still had very limited notions of how to program. I vividly remember being confused by the dot.access.syntax.of.javascript.

I went on to study physics at a local university. I wanted to be a research scientist. During that time I was employed by a university laboratory working in nanotechnology. However, I wasn’t a great student. I was more inclined to work on the interesting stuff in the lab than to study for classes. During that time I took my first real course on programming. First with FORTRAN because it was a requirement for my physics degree (Which I failed. Twice. As I said, I am a terrible student.) and later I took the proper introductory CS course which was done with C++, which I aced because at that point, because I knew the basic constructs of programming from FORTRAN and my physics studies had made me a very stubborn problem solver.
My point in telling you all of that isn’t to talk about myself. It is to demonstrate that I didn’t come into programming out of the blue. I had a history of exposure to it, and a technical background and education. I am unusually technically minded and a stubborn problem solver, but I’m not a gifted student or particularly hard working. I can’t be because…

I got sick. I couldn’t continue my studies. And I couldn’t work. And it was a real possibility that I might be sick for a very long time. I decided that it was time to switch from my dreams of doing research to doing something that I could possibly do from home. I decided to go into programming. I decided to combine another passion of mine, video games, with learning to program properly. I think that it is important to combine passions. It’ll keep you going when you are burned out on learning to program and having an actual problem that you want to solve is a great motivator. I played a number of games regularly that you could do some programming with. One was WoW, and you could write UI addon for it in LUA, but I felt that it wouldn’t be very marketable to learn LUA. Another game was Kerbal Space Program, which ran on the Unity game engine, which used C#.

I thought that since C# was very popular with business, and if I did want to get into game development, Unity would be a decent choice, I decided to run with it. I started out modifying an existing mod for Kerbal Space Program called Interstellar. Tweaking it to my own liking and fixing bugs. I learned about git and github, which was where the original author hosted his code. I also created my own mod from scratch called ForScience!, which was very popular. About a year after I started, the original mod author of Interstellar had abandoned the project, and my version was the only working version. I published my version and began work integrating the mob to play better with other mods. I released Interstellar Lite, and recieved my very first donation.

But all was not well. I hadn’t done enough research into my market. While C# was indeed very popular with business, there were relatively few jobs to go around in my area. And I had no degree, and my experience with C# was not with .Net. I had received literally no responses for my C# experience. I needed a wider net. And I was probably going to need to be remote, because my health still wasn’t great. And remote jobs tend to require more seniority for typical .Net shops compared to web dev it seemed. So I began to learn web development. I chose to do FreeCodeCamp and it held my hand just enough to stay on track. I created my portfolio site around 3 years ago. I already knew the basics of HTML and CSS, and learning javascript was fairly easy as far as the basic syntax goes. I continued with the program and maintaining my mods for a couple of years.

About a year ago, my health had started to improve, and I was looking to get a job very seriously. I also had developed relationships through the World of Warcraft community and there was an opportunity to work on a web app project. And so we started the shadowcraft rewrite, and I began to learn React and all of the modern javascript tooling/ecosystem. I wrote about that elsewhere if you’d like to read about that project. I took a year to complete that project. Until January of this year. But I started getting responses about my javascript, and in particular React, experience almost immediately. First technical was in May, and every month saw more and more interviews and technicals. But I was still very discouraged. I was getting past phone screens all the time, and the technicals would seem to go well, but for whatever reason, the opportunity would fall through. I can’t state enough how discouraging this was.

I was committed though. I knew that eventually someone would finally bite. That eventually my skills would be apparent enough that I couldn’t be denied because of my education and work history. I did get a few small contracts on UpWork during this time, which kept me going. And one day in Nov, I get an interview for a company and they hired me on the spot. I was immediately tasked with doing some technical planning for a new project and all seemed well. And then a week went by, and then another. The company had just died off.

I took consolation that I did make some money. But I was back to square one. At least now I had a company to put on my resume. I was still technically employed by them, just without any active project.
Then it really did happen. I get call from a client on Upwork, and they seemed confident that they have plenty of work. I’m hired on for a couple of weeks on a react project, and then they want me to come out to San Francisco on a react native project for a new client for a couple of weeks, and then new .Net project comes up. I’m asked to sign a contract directly with the company. I have more work than I know what to do with and I’m working with great people with tons of different technologies. I love it. I feel vindicated for all of those years of struggle.

I also learned that I could have been employed a few years ago. I was ready then. Some of my coworkers were very much junior to me. I just didn’t have the opportunity or the right marketing. If I was in a more populated area or if I had started with a technology that moves fast like the web, I would have been better off.
If you don’t have a degree and you are in a relatively low population area without much opportunities, your road is going to be more difficult than someone in a populated area or with a degree. Even if you are willing relocate at your own expense, companies are going to be more risk averse. If you don’t have a degree, trying to get a job at a standard business is going to be a very tough sell. Almost all of my big hits were from software/tech companies. Traditional businesses want that degree because they don’t understand that programming is a practical, creative skill, not just an academic one. If you are starting out, I would recommend a degree if you can afford one. I plan on finishing my degree as well, now that I have some income to afford it. It is worth the money, but you still need to learn to program. You do actually need to know a framework and the landscape of your chosen technology.

Some interesting things that I learned during the process of hundred real of applications (not just click > apply).

Research your market. Make sure there are plenty of jobs available for a technology that are looking for people like you before you invest in learning that technology. And try to pick a technology that is newer so that you don’t face as much competition with people with more experience than you. Don’t learn web forms, learn .net core. Don’t learn Swing, learn Kotlin. Don’t learn jquery, learn angular. That sort of thing.
My portfolio was just about useless. Both the website and my github account. I still think it is a good idea to have one. I learned a lot from those things over the years, but the recruiters, HR reps, and technical interviewers will hardly ever look at it. I was a surprised when an interviewer knew anything about me or my projects.

Keep your resume short. Don’t bother with an objective. Explicitly state each technology applicable. You don’t know how many times I was asked if I knew HTML/CSS or HTML5 or whatever even though I have javascript and react and other web technologies on my resume. Or my favorite. Do you know T-SQL when it clearly says SQL. The people reading your resume do not know anything about what they are hiring for. If you don’t say the magic words, they won’t understand. You are fluent in greek, they only know what is in their phrase book.

When they ask you to tell them something about yourself, keep it short. Like two or three sentences. For whatever reason, people seem to judge your experience more optimistically if you leave out details. If you go into more detail, you risk exposing that you don’t know something they think is critical. Don’t ever pass yourself off as a junior, pass yourself off a yourself. Don’t say you have more experience than you do, but don’t let on that you don’t know something. This is contrary to all of the advice I’ve seen. Everyone says that you should say when you don’t know something and that they are looking for curious learners.
This is bullshit and I don’t know where it comes from, but you are being hired as an expert. You don’t want a junior surgeon, you want the surgeon that has done the thing 10000 times with zero complications. This is more true for the non-technical people in the hiring process. To HR and the CEO, you should be a god like miracle worker with infinite knowledge. To the technical director, you may not be sure, but this seems related to that and you have experience with that, and would be excited to work with it.

Get your first job, then think about your dream job. Don’t do those practice sites. Learn real skills. Learn a framework/library and use it extensively. Ignore the vanilla javascript peeps. You aren’t being hired because you know javascript event loop. You aren’t being hired because you know how to bind/call/apply. You are being hired because you can make a button turn blue or track down a mutation bug in Redux while the website is on fire. You are being hired to solve actual problems and build features, not because you know the algorithmic efficiency of bubble sort. They want you to be fast on an interview, even though you can take as much time as you need. The way you get fast is by doing it a lot, and to do that you need to build, build, build. You need to write tens of thousands of lines of code. As much as you can. You ideally need to work on a project that is actually used by people. You need to know what it feels like when things go wrong and the stress rises. You need to get stuck. Like really and truly dumbfounded. You need to learn how to claw your way out of the hole when you are hopelessly lost. You need to learn to learn.

I think that anyone can learn to program. I also think that it takes a special type of person to do it well, and enjoy it. I think that you need a enjoy hard problems. Maybe I am projecting, I don’t know. But I think that is very unusual for the average joe to just pick up a book on programming or follow a course and learn to program well enough to be employed. I think I’ve read that something like half of these bootcamps are attended by someone that already had a job programming. I am very skeptical that someone without connections learns to program in a few months and is making tons of money. I think that it takes years of exposure to be ready to work on all but the most trivial projects. I think that the reason you see these articles about getting a job 3 months after learning to program are exceptional exactly because it is exceptional. If it were routine, there would be little value in sharing the story because it would be a story we’ve all heard. And I’ve seen more than a few where it was clear that the person had previous exposure to programming.

If you made it this far, firstly, thanks, as I know this was quite lengthy. And secondly, my point isn’t to discourage someone. I just think that a lot of media tried to play it up like it is easier to do than at least my experience. If you look around, you’ll probably see that it is far more common to hear about someone having a hard time getting their first dev job, even from CS degree graduates. But it is worth it, at least to me, having gone through the process. I work from home now. And I get to learn new things every day, which I love. I found my place, and I hope that you can find yours.

The Shadowcraft Rewrite

It has been a couple of weeks since the launch of the new frontend for ShadowCraft and I thought that I would talk a bit about the project and what went well, as well as some of the mistakes we made along the way. (Here’s my character as an example if you don’t have one.)

So, what is ShadowCraft? Well, strictly speaking, it is a Python script that uses ‘steady state’ models to approximate the in-game performance of rogues. We call this the shadowcraft engine. This is in contrast to the more common approach used by projects like SimulationCraft, which model player characters by simulating every detail of a player in combat. Because of the randomness inherent with all of the special abilities, one must run not just one simulation, but actually a great many and then average those simulations. This is a very computationally expensive proposition. SimC is rewritten in C++ by very competent developers, but nonetheless, it is not uncommon for players to have to run SimC for hours to check all of the various effects that they want to compare. One must be very careful to keep the number of permutations in variables low in order to even complete in a reasonable timeframe.

ShadowCraft instead makes certain assumptions and uses averages to model characters. For instance, whereas a simulation would simulate swinging the players sword ingame thousands of times, ShC would instead model that as the average weapon damage over time. To get the total damage over the time frame of a typical fight, one only need the total time for the fight, and it is a inexpensive calculation to find the result. That is ShadowCraft in a nutshell. If you are familiar with spreadsheets, it is much like that. In fact, it started out from spreadsheet models. This allows for very quick comparisons, at the expense of accuracy and quite a bit of developer effort. This make it ideal for things like talent or gear comparisons after you know the behaviors in game. Generally you learn optimal rotations with simulation and then bake them into the models to compare gear, gems, enchants, or talents, or whatever else.

Now that we understand that. This project that I am talking about today concerns the website that interacts with the ShadowCraft engine and presents the results to the user. Previously, it was a standard Ruby on Rails web app. Server rendering, controllers, jQuery, Coffeescript, the whole nine yards. And it was old. It had a complicated build system, and you really needed to know Rails inside and out to work with it.

Now I know that there are Rails developers out there that could have probably made an easy time with the project. The problem was that over and over we saw contributors, many of which were relatively new to development, that would come in, express interest, and then see the code base and the build, and quietly slip away. No one makes money from ShadowCraft. There is no patreon, we are hosted by the graces of mmo-mumble, and the little bit of ad revenue generated goes to offset the hosting costs. I’m not saying the project needs money, just that there isn’t much money to be had here, and certainly none for development. It is very much the result of a few passionate and committed players that keep it running. So having potential contributors walk away is a problem.

We needed something simpler. And one that hopefully that leveraged javascript and it’s much higher popularity and availability of developers. We chose a very thin Flask backend, which also leverages the fact that the engine runs in Python anyway, and it is very popular beginner developer language. For the client, we decided on React. Mostly because it was popular and that it looked really easy to use. We are VERY happy that we chose React. It has been lovely to work with. There is no magic, and very little to learn. Exactly what we were looking for. The rewrite was mostly developed by Tamen and I, and neither of us had much experience with modern javascript front end development or javascript in general and it worked out just peachy. I can’t say enough how happy we were with this choice. (I also sorta fell in love with javascript in the process. /swoon)

We also utilize modern frontend tooling like Webpack, ESLint, Jest, etc. It also uses Immutable and Redux. I think we still have a ways to go with our build system, and I would definitely love to increase the test coverage, but we have all the most important bits covered by tests.

We love Redux, we came to it after lifting state and passing down handlers and sort of reached for it naturally. This approach worked great for us. We found out first hand why we needed it, and it was a very natural transition for us. I will say that it probably took me 2 or 3 weeks to understand it in practice. And dealing with nested state is still a bit of pain point, but after those growing pains, I love it. It has really had an influence over how I think about development in general, not just for React apps.So what didn’t go so well?

The most obvious mistake that I think we made was doing a ground up rewrite. I was thinking 3, maybe 6 months, and it ended up being 12. The front end fell behind as a result, since it was basically just Tamen keeping the Rails app on life support and the engine fell behind with the loss of Fierydemise. Mystler has done a remarkable job taking over the engine, but he could have used more support. Mostly I am talking about the Outlaw model, which has been useless for the entirety of the expansion. (Something I intended to help with, but got sidetracked with the front end, which is why I think it was a mistake.)

I think that we did a disservice to our users with the rewrite instead of simply componentizing the rails app and making a slow transition over to the new stack. I’m less sure that we would have ended up with the structure and some of the features that we have now, and maybe the current product is better than where we would have reached in the same timeframe. However, I do feel that the live app ended up not getting enough attention. But who really knows how it would have played out. You never really know with these things.

Lesson: I will much more carefully consider any rewrite notions that I have for any future project.

Immutablejs is still a controversial decision that we actually made quite late in the process. We had trouble with mutation in the artifact component (and few other odd places), and there was a lot of articles about how to help enforce immutability and loads of recommendations for immutablejs. I’ll be the first to say that I bought the arguments, and I was even quite enthusiastic about it at first. But after getting into the transition, that is when I discovered too late that you can’t really do inheritance with immutablejs. Even though I came from the C# world, I don’t really think everything needs to be OO, but sometimes, including on this project, we have coherent objects we want to deal with. and immutablejs makes that a pain.

It also swallows any of the type/shape information about your objects that your tooling may have been able to provide before. Which was a huge let down for me. Immutable also reaches in a infects the entirety of your code base, which I found very distasteful for lack of a better description. We are now married to the library and it will be extremely painful to remove should it come to that point. Also being a fairly late decision, it probably ended up actually delaying the release for at least a month, maybe two, because we were tracking down all the errors that resulted.

The library also adds another layer of knowledge needed to be productive with the project. I think we could have done without it, and simply relied more heavily on testing and tooling to detect mutation. Many of our problems with mutation were simply the result of inadequate formal testing. I believe that if we had written testable code from the start we would have avoided most of these problems. This should probably be a separate lesson.

Lesson: Don’t be so quick to follow the pack in JS land.

The person writing probably doesn’t care the same way about type information, tooling, or developer experience as you do because all they’ve known is vim/emacs and js/python/ruby. (Which is fine. I just have my own preferences and those factors are important for me.)

So that’s it. I think that covers the big stuff. There is still plenty of work to be done. I’d like to refactor and reorganize the project to make it a bit clearer, and to improve testing. We also have some ideas for features that got put off for the initial release, but it is nice to see the project in maintenance. It was by far the largest project that I’ve worked on significantly. And by far the most code that I’ve written for a project. It was a very fulfilling accomplishment for me.

It would not do to end this before saying that this project would not be complete with Tamen. The guy is a beast. I think something like 4/5ths or more of the code was written by him alone. It was fantastic working with him on this, and I continue to learn and be inspired by him. Thanks, Tamen. You are awesome. If you have the chance, be sure to thank him for his tremendous efforts.

And if you are looking for a project, and/or you love world of warcraft, please consider contributing. We have tons of work, and friendly people to work with. And if you are new, I can tell you it is a great way to get some experience with development. Just message me on ravenholdt or tweet me how to get started.

Optimizing the Landing Page

So now that the website has been converted over the Gatsby. I have been looking into performance.
One of the neat things about Gatsby is that it will render your app out statically and then load the rest of your app after the initial render. There is this interesting optimization you can use with your internal links to benefit from client side routing with Gatsby. I had seen it mentioned in the docs for Gastby, but it didn’t click with me at the time. However, I came across a post by Scott Nonnenberg who mentioned that one can avoid a full refresh of the page by utilizing the Link component provided by Gatsby, as opposed to the regular html anchor tags which I had been using. And finally my nagging question was answered as to why my site didn’t feel as snappy as the default starter. Swapping those a tags to Links made a huge difference because now each route uses cached resources lazy loaded by Gastby. Awesome Sauce!

With that handled I put my efforts on optimizing loading speed for the initial render. I knew that most people would be hitting my landing page first so I might as well start there. I first headed off to PageSpeed Insights, which is a tool offered by Google to analyze page performance. There are many other similar tools, but this one was pretty straightforward, and since I have no preference so I just went with it.

sample report from google after a little optimization

One of the messages was that the lightbulb ‘hero image’ on the landing page which I had been using was 4.3MB. which is obviously a lot to be loading on the initial render. And a general first approximation for improving performance is simply doing less work. And downloading a smaller file definitely is less work. I played with it a bit, but Google kept wanting a smaller image. And I’ll admit I was afraid of ending up with a very weak image with high compression. I put the image through gimp and set the output to 80% quality and reducing the resolution to 1920×1080. Turns out it looks fine even at 1080 on my monitor. I ended up at 109kb, which is pretty good IMO.

The blocking resources will have to wait for now, until I can get rid of the dependency on Bootstrap. And I’m honestly not too worried about the caching thing. I plan on uploading every week, and it is currently at ten days. I don’t expect there will be that many repeat visitors anyway. Let’s be honest. I basically talking to the void here. 😛


Wow! I haven’t blogged in years. Since about 2005, I believe. My old blog is now long gone. Maybe one day I’ll resurrect those old posts from the internet archives. Not that I think there is much value in it. However, I do believe it has posts about my first attempts at javascript. Who knows.

I have completed the transition of my blog over to Gatsby. One of the reasons I chose Gastby is that I can use my normal React workflow, but it will also statically render all the assets for very fast load times. I had been using manual html files with html imports to handle things like headers and templating. While HTML imports worked fine for the most part, I just couldn’t see myself doing much more than a very basic website with it. It doesn’t seem like HTML imports is really going to catch on any way (probably for good reason).

I knew that I didn’t want to tie myself to WordPress, which would have required me to get a domain name or link away from my GitHub profile page. Nevermind the issues with security. And if I was going to go with a server rendered site, it would probably be on Azure. There is also Jekyll (I misspell it every time it seems), which is what GitHub Pages recommends, but I didn’t want to hitch myself to the Ruby wagon. I already have enough tools to keep track of.

But Gatsby…Gatsby has worked like a charm. I am very happy with the results so far. I am hopeful that this will work out in the longer term. I’ve had tremendous fun putting it together. And maybe this can be an outlet for my many random rants instead of them being squirreled away in random social media posts.

Time will tell.

Yesod Dev Container using Docker (on Windows)

So I’ve been learning Haskell and I believe I’ve learned enough syntax to begin working with more non-trivial applications, and since javascript/web applications are my primary domain, a web framework makes sense.
Throwing a bit of a monkey wrench into this process, I’ve run into difficulty running a few haskell projects on windows. Plus I don’t want to clutter up my personal machine with a bunch of dev tooling. Using docker is a great way to isolate your dev environment, not just deployments.
You probably want code, so here it is. I won’t ramble until afterwards.

# Dockerfile
FROM fpco/stack-build
WORKDIR /yesod-test/
COPY . .
RUN stack setup
RUN stack install yesod-bin --install-ghc
RUN stack build

Using the haskell docker image as the base image wouldn’t compile due to some error with blaze-html or something, which I didn’t bother to investigate. (There was basically no other information in the error output but the name.) The fpco/stack-build image is apparently has all dependencies needed to build packages on stackage, which is exactly the source of our deps, so this is close to ideal, at least for development.

This uses the fpco stack image, sets the current directory to the folder where we will mount all of the files for the project (yesod-test). From there it is standard yesdo commands.

The port exposed must be the same as the one set in the `command` in docker compose. The yesod dev process ignores any of the project config vars, you need to set the port using the cli flag, as far as I know.

# docker-compose.yml
  build: .
  command: stack exec -- yesod devel --port 3030
  stdin_open: true
  - '3030:3030'
  - .:/yesod-test
  - /yesod-test/.stack-work
  - /yesod-test/yesod-devel

I’m still learning docker as well, which means I still constantly run into issues with it, and quite often the images or configs that I find are out of date. I was able to get a dev container to compile with this process. The volumes also mount the project files inside the container so that the file watchers can detect changes and recompile the project and automatically refresh the page like any sane modern development workflow.

Caveat, that there appears to be some issue with file watching with Docker for Windows. The docker devs seem to think it is resolved, but it is not. I was able to work around it on a Next.js project with a webpack polling hack, but that specific solution isn’t applicable here but I assume something similar could be done. Until this issue is resolved with Docker, the experience will be less than ideal on Windows.

Note the matching port numbers. All the volumes are build artifacts we want to persist or project files `stdin_open: true` is needed for some arcane reason.

It is frustrating to run into so many issue on Windows with Haskell when going beyond trivial exercises with GHCI. I’ve thoroughly enjoy learning so far with The Haskell Book but moving beyond exercise has been a trial of patience so far. I didn’t have much luck with Miso (ran into issues with GHCJS, undocumented nix dependencies) or gi-gtx-declarative (couldn’t get everything installed without errors) despite being very excited about those projects.

I’m sure much of this is my inexperience, but I hope someone at least found the docker configs helpful.

Feel free to tweet me about how I’m wrong. (No really, I love discussions.)