Learning Rust Modules (Talk)
My earlier post on “Learning Rust Modules” prompted the Columbus Rust Society to invite me to talk on the same subject. This is a recording of the talk I gave on 2015-09-23.
CSS is Not Composable
I’ve been a .NET web developer for more than 12 years now. In that time, I’ve had quite a few projects at a mix of consulting gigs and various jobs. Generally, I am a full-stack developer meaning my work ranges all the way from the database, through the business logic and into the front-end html, CSS and JavaScript. The projects I work on are typically line of business web applications, though I spent a little bit of time at an interactive firm and my current job has me working on a public facing CMS and LMS. None of the projects I have worked on ended up with clean clear style sheets despite a variety of approaches to them. Inevitably, they started out not too bad and slowly devolved into a horrible mess.
My CSS Background
Before everyone tries to say that the teams and I just didn’t understand CSS or know how to use it. Let me say I am quite knowledgeable about CSS in regard to both technical and best practices respects. I understand the box model, selector specificity, floats and clearing, CSS 3 selectors and all the rest. I’ve read extensively online about CSS best practices on sites like CSS-Tricks, A List Apart, Smashing Magazine and many others. I’ve tried to understand OOCSS, SMACSS and BEM (which wins the award for ugliest naming convention). On different projects I have tried different approaches, including “anything goes”, semantic CSS and most recently trying my own flavour of OOCSS. I’ve recently gotten extensive experience with Bootstrap. I think it gives a good insight into how a lot of real world CSS is done. With all those approaches, the CSS becomes a mess and a pain. I’m open to the idea that I just haven’t found the “right” way, but I would argue that, after all that research and experience, the problem isn’t with me. It is either with the technology or the community’s inability to explain the right way to use it.
I used to be a true believer in semantic naming. In practice, that often means content based naming and lots of words from the business domain in the class names. However, that approach leads to many separate classes that have the same styling. Recently, I’ve been working really hard to try and create functional class names. That is quite difficult because there just aren’t enough page layout terms for all the classes and design elements that are needed. I found this approach eliminated the duplication of styling on many different classes, but there was a deeper problem. It was now very difficult to get things to layout correctly without littering the HTML with tons of classes, many with vague, easily forgettable names. In my reading, it seemed like this wasn’t supposed to be a problem. In fact, the goal was allow composable layout classes. I read that:
[…] a latest news module found in a sidebar should not be defined by its current place in the sidebar. It should be movable to a main content area, another module, the footer, so on and so forth.
Yet, I found the exact opposite was actually the case. When I used the OOCSS approach I ended up with design elements that only worked in one context and I either needed a different design element or lots of extra classes to express each of the different places it could appear. After more research, it seemed I’m not the only one who found this. Ben Frain expresses the same difficultly in his post “OOCSS/Atomic CSS are Responsive Web Design ‘anti-patterns’”. Now, to be clear I don’t think this is the fault of OOCSS. I think this is an inherent issue with CSS.
A Real World Problem
As an illustration, on my current project I was attempting to follow my new OOCSS like approach and found myself stuck. This was the issue that clarified for me what was wrong with CSS. The problem wasn’t that I couldn’t come up with any classes and CSS that would layout the page the way that was needed. It was that there was no way to make what I thought was a reusable CSS “object” that works when combined with various other design elements. It seemed to be necessary to have classes and styles for each specific context. In an attempt to learn what I was doing wrong, I posted the problem to Stack Overflow, asking what the “CSS Best Practices for decoupled modules” are? I worked hard to clearly express that the problem wasn’t that I couldn’t create classes and styles to meet my needs, but that I couldn’t do so in a composable way. I think I succeeded in expressing my question. In the next section, I’ll show a variant of the issue I used in my question. First though, let’s look at the answers I received.
There was only one response. I do appreciate the answerer taking the time to write up a lengthy response. It helped to further my understanding of the reality of CSS. However, the answer amounted to “that’s not how CSS works”. Additionally, he went on to describe a number of “best practices” that were just an explanation of how CSS works. It is a pet peeve of mine that the CSS community seems to be the only development community that thinks describing how the technology works is a description of best practices. In every other development community I have been involved with, best practices are those approaches and procedures an expert in the technology will find most effective. They assume the audience is already aware of how the technology works but that they need guidance on which of the many ways of using that technology would be most effective.
Modules that aren’t Modular
The example I used in my stack overflow question was an instance of what many designers call a “module”. A module is any design element that has a standard box appearance, but can contain a variety of contents. A common example of a module is the aside. Typically asides are presented as floating boxes of a fixed width with certain margin and padding. Most commonly asides contain paragraphs of text. However, some asides might also contain lists, quotes, floating images or even a little form (for example, an aside might provide a contact form).
The problem arises with the bottom padding of the module. The designer wants to ensure that there is always at least so much padding. However, some elements have their own bottom margin. That margin combined with the padding causes too much white space at the bottom of the module. We can’t simply adjust the padding so the total white space is correct because the module can contain different elements with different bottom margin. This is a common enough problem that CSS-Tricks describes 7 ways to solve the problem in their article “Spacing The Bottom of Modules”. The last solution they land on is to remove the margin from the last element in the module and its last children like so:
However, that isn’t really composable either. The problem is that the last child is the last child in the DOM, but may not be the last child visually. This can happen when there are hidden or floating elements. The visually last element can even be dynamic. Consider a module containing a floating element in a fluid responsive layout. The floating element needs left and bottom margin to provide space between it and the wrapping text. When the viewport is narrow, the text may be the element in contact with the bottom of the module. However, as the viewport gets wider, the floating element may become the element in contact with the bottom of the module. Then the space between them will be the sum of the module padding and the floating elements margin. That is more space than desired. One could use a media query to remove the bottom margin of the floating element at this point. However, because the width this occurs at is dependent on the exact size and position of the floating element as well as the exact text and font, this would have to be done on a case by case basis.
This is just one example of situations like this. There are many other ways this happens. As another example, imagine a responsive grid layout. It could be the case that at each breakpoint a different element becomes the visually last element in a container and each one has different margin. Correcting that requires a very complex set of breakpoints in one’s style sheet. There is no general, composable, way to deal with these issues. The only approach I can find is to deal with them on a case by case basis on each page or particular arrangement of elements.
What’s My Container Width?
Lest you imagine this problem is limited to margin and padding, consider the humble article image on a fluid site. Imagine in our design that at larger viewport sizes the image should float to the right with text flowing around it with a certain amount of margin all the way around. Of course, at some point as the view size gets smaller, we will want to stop floating the image and change to a centered block. That should be simple enough. We can use a media query to change the layout at a certain breakpoint. If we know the width of the article area and the width of the image we can easily decide on a reasonable breakpoint that prevents the text area to the left of the image from being too thin.
However, what if those two implicit assumptions don’t hold? Imagine that, on our main page, articles are displayed 100% width, less some margin on the sides. On category pages, articles are displayed at 80% width to allow for a 20% navigation bar. Now the width of the article is context depended. We can imagine more complex site designs with more than two different article width contexts. Alternatively, imagine that the images in different articles aren’t all the same width. We’ll set a max width and height so they never get too big, but depending on the image size and aspect ratio we could need a different breakpoint for every image. What is really needed is the ability to change the style based on the size of the container or better yet, based on the amount of space between the element and the edge of its container. That is how a designer would express the rule. He would say, when the space for the text next to the image is less than a certain amount, rather than floating the image center it above or below the text.
CSS is Not Composable
I believe the root of the problem with CSS is lack of composability. There are minor annoyances like, the lack of variables which can be addressed by preprocessors such as Less and Sass. Yet the challenge of CSS remains. If CSS was composable, it would mean design elements could be placed next to one another or nested or combined on the same element without unexpected consequences. Without requiring special CSS for the particular combination of design elements or particular page. The need for CSS for each particular combination produces coupling. Design elements are no longer fully independent. This coupling, just as in object oriented design, leads to difficult to understand and maintain code that lacks flexibility. Composability produces flexibility because it enables rearrangement in a myriad of ways. I long for a web design language with that power and elegance.
Learning Rust Modules
This post is based on Rust 1.2.0 and I am just starting out with Rust, it is possible I misunderstood something.
The other day, I sat down to write my first bit of Rust code ever. I was working on a simple kata and many other people would just whip up something with all the code in one file. However, working as a C# developer for many years, I am in the habit of organizing my code into namespaces and separate files. I was totally stumped for a while on how to do this in Rust. The documentation of modules wasn’t immediately helpful. I later figured out that certain key sentences did in fact explain how modules work in Rust. However, coming from a C#/Java way of doing namespaces/packages, they weren’t explicit and direct enough to flip around my thinking. Now that I’ve figured out how modules work, I thought I’d share what I figured out.
Review of Namespaces
C# namespaces and Java packages are really very similar. In this post I’ll focus on C# namespaces, because I’m a C# developer, and only mention Java when there is a difference. By convention, we organize and name our code files by the namespaces and classes they contain. But really there is no correlation between the file structure of the code and the namespaces and classes. We are free to name each file totally differently than the classes it contains and put it in a directory that has nothing to do with the namespaces in it. Indeed, we could combine our code in a single file if we so chose. When compiling, we are really providing a list of files to compile together. Typically our IDE hides that from us by either making a list of the code files in a project file, or simply assuming that every code file in a project directory should be compiled together. Although we typically don’t think about it this way, namespaces aren’t really entities, but are just a way of creating really long unique names for classes. So the class MyNamespace.MyClass
could logically be thought of as being one long class name MyNamespace__MyClass
with some compiler help to make it easy to refer to by figuring out the first part of the name (note, class names aren’t actually changed this way). In fact, in the compiled code namespaces exist only as long names for classes and every reference is fully qualified with its namespace. With that refresher, let’s look at how Rust does modules.
Rust Modules
Rust modules do not work like C# namespaces or Java packages. First, in addition to organizing classes like namespaces, they can also contain static variables and functions . In this way modules are similar C# static classes or to a Java class containing only static members. Second, Rust modules aren’t unrelated to code files the way namespaces are. When we put all of our code in one file, they look very similar. For example, if we were implementing a math library, some of our code might be:
However, if we would like to spilt our code into multiple files then they work differently. Rather than declaring the module in each file, we declare the module without a body in the parent Rust file and the compiler “includes” the code of the corresponding file. The pulling in of the other code file feels like C style #include
although it isn’t being handled by the preprocessor, so it isn’t a simple textual include with all the gotchas that come with that. The Rust compiler will look in two places for the file to include for a module. It will look in the directory of the parent code file for module_name.rs
or module_name\mod.rs
. Thankfully, having both files is a compile error. The second form, using a directory, allows one to have further modules nested inside the given module. I imagine most people will use the first style whenever possible. When compiling Rust, you are really only providing one main file to compile and it finds the rest by including them as modules. If we split our math example up using this technique, we could get:
Notice that lib.rs
contains the declarations of the number
and trig
modules. This is what actually creates the modules. The other files contain no reference to the modules. Their contents are in the module by virtue of the Rust compiler including them because of the module declarations in lib.rs
. I’m trying to withhold my judgement, but coming from a C# background this just feels weird.
One Type, One File
If you’re a C# developer looking at this, you’re probably thinking “but Complex<T>
and Vector<T>
are still in the same file, how do a separate them?” It is actually not possible to spilt a module across files in Rust the way one can split a namespace in C#. So to put them in separate files, they would need to be in separate modules. However, there is an interesting workaround for this. Notice that we specified our modules to be public using the pub
keyword. C# namespaces aren’t public or private, only the classes in them are. Rust also allows its equivalent of the using
statement, the use
statement to be public or private. When use
is private, it functions very much like a using
statement. When it is public, it allows one to incorporate stuff from one module into another module. These features allow us to put types into their own files and hence own modules, but expose them to the outside world as if they were together in one module. Doing that to our math example would look like:
Notice that in mod.rs
the two modules are not declared with pub
. They default to private, so they are not visible as sub-modules. Then we use public use statements to include Complex
and Vector
from their sub-modules into the number
module. Now we can use them as if they were in the number
module.
Crates
Up to this point, we have talked about modules within your own code. However, it won’t be long before you want to use modules written by other people. These are packaged in what Rust calls “crates”. A crate is a library or executable and is the equivalent of the .NET assembly concept. The closest equivalent in Java would be .jar
files. They are also the unit of publishing packages like Nuget packages in .NET. You can browse the published crates at crates.io. If you wanted to use the primes
crate (to pick a random example). Simply add to your Cargo.toml
file the lines:
Then from the code you import the crate with:
This exposes the contents of the primes crate to your code in a module named primes
.
Something important to know, that isn’t stated in the docs right now, is that many crates have dashes in their names, but dashes are not allowed in module names. When using one of these crate you use the name with dashes in the Cargo.toml
file like media-types = "0.1.0"
. However, when referencing the crate from code, replace the dashes with underscores. You can also use as
to bring any crate in as a different module name.
Note that the dash in the crate name is replaced with an underscore to give a valid module name. This also shows bringing the module name in as “mt”. Older versions of Rust handled this differently, so you may see some examples with the old syntax that uses double quotes.
Modules are more than Namespaces
Earlier, I mentioned that modules, unlike namespaces, can contain static variables and functions. In this respect, they are like C# static classes or a Java class containing only static members. All of these have a lot of similarities to an instance of the singleton pattern. We have already seen an example of static functions in a module in the sin
and cos
functions. Now, let’s looks at static variables. They enable a module to have and keep state which can be private to the module. Static variables are declared with the static
keyword very much like let
.
Notice the special lifetime 'static
that all static variables have. In addition to the special static lifetime, they have a number of restrictions that let bindings do not. Their type must always be specified. Accessing a mutable static is an unsafe operation because other threads could be modifying it at the same time. Their value must be initialized to a constant expression. Finally, they can’t implement Drop
. These restrictions ensure thread safety and avoids the complex issues that C# and Java have with static initializers and clean up of statics on program exit.
Paths
I’ve already shown several examples of the use
keyword. It is important to note that it brings into scope the final name in the path. That is the name after the last pair of colons. This differs from the C# using
keyword which brings in all the contents of the namespace listed after using. Given the ability to re-export names with public use statements it makes more sense for it to work that way. Like with using statements, the module path of use is always relative to the root module. However, when using paths other places in code they are relative to the current module. There are a number of useful options available for the use statement. I encourage you to read about them in the docs for use. In addition, you can use the as
keyword with use in the same way as with extern crate.
Edit 2015-08-30: Corrected explanation of dashes in crate names to reflect current version. Added “Paths” section based on feedback from the Rust subreddit.
Unleashing C# Generic Constraints with Extension Methods
A number of of times in my career, I have found my work frustrated by the sometimes arbitrary limitations placed on generics in C#. The addition of generic covariance and contravariance was a big step forward in that regard. Still, there remain many frustrating limitations. The fact that you can’t use Enum
or Delegate
as a generic constraint can be worked around using packages like ExtraConstrains.Fody or UnconstrainedMelody. However, extension methods also provide a little known way of working around some limitations. It is so little known that I couldn’t find a blog post that discussed the technique while working on this one (though I think I recall reading one once). Indeed, these over 2 year old stockoverflow.com questions “Is it possible to constrain a C# generic method type parameter as ‘assignable from’ the containing class’ type parameter?” and “Constrain type parameter to a base type” had no answers showing this approach until I answered them while writing this.
An Example
Imagine we create a generic pair class that we will use any time we want to deal with a pair of values that may or may not be of the same type.
Then we might find that we sometimes want to know if the two values are in order. So we imagine we could write a method to tell us if that is the case.
We’ll quickly realize that this code doesn’t compile at all, because we aren’t allowed to add generic constraints to a non-generic method.
Another time, we think it would be useful to be able to apply a function to both values. So we attempt to write the apply method. We will need to constrain the type of the function to accept both the first and second value.
But again, we are thwarted by the compiler. In both cases, the underlying issue is that you can’t further constrain the type parameter of a class from a method.
Lastly, we might imagine it would be nice to have a method that swapped the first and second value. Of course, that only makes sense when they have the same type. It is very difficult to envision how one might write this method as there is no generic constraint for type equality in C#.
Extension Methods to the Rescue
In all the above situations, an extension method can easily be used to work around the limitations of generic constraints. That looks like:
Notice in the swap method how we are able to specify the first and second types as equal by simply using the same type parameter for both.
Not Perfect
This approach sometimes leads to situations where the type parameters can’t be inferred and it is necessary to specify duplicate type parameters since there is no way to specify one type and have another inferred. It also doesn’t address other limitations of generic constraints, like the fact that you can’t specify a constructor constraint with parameters. Still, I hope this will be a useful trick you can add to your toolbox.
Advice for Open-Source Projects
Recently, I’ve been comparing JavaScript build tools and their plug-ins. At the same time, I’ve been checking out static site generators like Jekyll, Hexo, Middleman, Metalsmith, DocPad, Champ, Assemble and others. For a more complete list, see StaticGen.com and this survey of .NET static site generators. However, I don’t really want to talk about static site generators today.
What I want to talk about is all the things that go into an open-source project besides the idea and the code. Now of course, one should have a cool idea for an open-source project and someone is going to have to write the code. I absolutely care deeply about quality, clean code. Yet with an open-source project there is a lot of other stuff that goes into it. Often times, it is all that other stuff that makes an open-source project great and gets people to use it. Now I can’t promise you that if you do all these things people will use your project, but it will certainly help.
All Open-Source Projects
There were a couple things I saw that we all need to do better at, regardless of whether one manages a massive project with hundreds of contributors or just wrote some code in a hour and posted it on GitHub.
Clearly Indicate the Status of the Project
The most important thing you can do with any code you post online is clearly say at the top of the read me or other most visible place, the status of your project. This should answer questions about the current state of the code and about the likely future path of development. Here are some ideas for statuses one might use. Though a couple sentences of explanation in addition will always be helpful.
To describe the state of the code:
- Works for Me or No Warranty: Indicates this is some code you threw together, it solved your problem or met your goal, but there has been no effort spent making sure it is suitable for others to use.
- Sample, Example or Starting Point: The code demonstrates how to do something, but shouldn’t be taken as production code. It should probably be incorporated into other projects as their own code that they will own and further develop in the future.
- Pre-release or Beta: The project has gone through multiple rounds of active development with a goal of reaching a stable release version, but is not there yet.
- Stable/Release: The project has reached a stable point where there should be a minimum of bugs and the necessary features are present. Use this even if there is a previous stable release, but you are working on a new version that is in beta.
To describe the future path of development:
- Active (with Date): This indicates people are currently actively contributing and addressing issues and questions on a semi-regular basis. The current date should always be included with an active status and updated at least twice a year so that people know it is true and not just a statement that hasn’t been updated correctly.
- Inactive: The project is not currently being maintained, but there are still people who would notice if someone reported a horrible bug or submitted a pull request. No guarantee is made that they would be addressed though they would at least get a reply.
- Hiatus: The project is currently inactive, but there is an intention that it will return to an active status in the future. An idea of the time-frame is helpful here.
- Ignored: This code is not being maintained, issues, questions, bugs and pull requests will not be looked at.
- Legacy: The project was once active and may have had a reasonable community. There is still some work to fix important bugs and answer some questions, but no major features will be added.
- Obsolete: The project uses technologies or versions of those technologies or approaches the owner and contributors think are obsolete and not worth further development. The code remains available for existing users or new users who are for some reason stuck on the old technologies. It may be helpful to combine this with a status like inactive, ignored or legacy to give a better indication of what users can expect.
You might imagine that large open-source projects with slick websites are exempt from needing to describe the project status because their website indicates the amount of effort being invested into the project. However, it is not uncommon to come across a beautiful open-source project website that was created four years ago and the site and project have been ignored ever since.
State a Clear Value Proposition
Explain in a short paragraph what your project is and does and why people would want to use it. Don’t assume other technologies you reference are known to the reader. They may have stumbled across your project even though they work in a totally different technology stack. When referencing other technologies, provide a link and a couple word description of what it is. Make sure it is clear what your project does. It is very easy for you, who knows the project inside and out, to not make this clear, because it is so obvious to you. Finally, include why someone would choose your project over other options. Try to avoid purely technical features like performance. Focus on features that are distinctive and not shared by most projects like yours. If you really want others to use your project, think of this as your sales pitch.
Committed Open-Source Projects
The above two things are what I think every bit of open-source code thrown up on the web needs to do. But what if you are actually wanting to start an open-source project that you hope people will use? Or what if you become a core contributor to a project you want to see succeed? The follow is my advice on how to make that project a success.
Recognize the Commitment and Make it
Creating and maintaining a successful open-source project is a lot of work. It is often as big as any project you might tackle at your job. Furthermore, you will be called on to do tasks you probably aren’t responsible for at work, like documentation and website design. There will be important bugs that if not fixed in a timely manner risk alienating your user base. There will be lots of questions and requests for help, many of which will be pretty “stupid” questions. All that will be done in your free time when you could be doing something else (unless you are one of the lucky few who is responsible for an open-source project at work). While you are spending your free time, there will be long stretches with little to no positive feedback. All that will take a lot of commitment. The most important thing you can do before starting an open-source project or becoming an important contributor to one is to carefully consider all the work it will entail. Don’t shy away from it, you need to face it upfront so when difficult days come you will know you anticipated them. If after all that, you still want to do the project. Then make a firm commitment to do the project and stick with it. The most successful open-source projects generally have a core team of contributors who were very committed for a very long time.
Don’t try to be All Things to All People
Given that this is an open-source project and not a product one is profiting from, it is important to limit the scope and focus on the important things. In large part, that means you don’t need to cater to every option and work style your users might have. It is good for your project to be opinionated and focused. If users don’t like those aspects, there are lots of other projects for them to choose from. It is much more important for the project to be excellent for a small dedicated user base. Trying to include as many users as possible often leads to a system that is incomplete and buggy. For example, the Pretzel project is trying to create a .NET port of the popular Jekyll static site generator. However, at the same time they are throwing in support for the razor template engine. Now their effort and focus is divided and they have twice as many things to test and document. I think this is leading to reduced quality. For example, when I last downloaded the project, the sample razor template site didn’t even compile. Once a project has successfully completed its core and has a growing user base, then it can expand to other functionality if it makes sense.
Docs Are as Important as Features
I see this all the time, a project is described in glowing terms and lists really cool features, so I go to use it only to discover there is no documentation except for one or two blog posts that barely count as an introduction. What the majority of potential users do at that point is look for other options. That is why your documentation is just as important as your features. Commit yourself to writing the documentation for a feature as soon as it is in a state where it is ready to be used. Also, don’t forget to update them as features are changed. That way the docs will always be up to date. The documentation needs to be a website or wiki or read me available on the web. If users have to download something to read the docs, a certain percentage of them won’t bother. Also, a bunch of blog posts don’t count as documentation. Blog posts are great for introducing new projects and features and for promoting a project, but they aren’t documentation. They can’t be updated to match the current version. It is incredibly frustrating to be referred to a blog post as documentation only to find that the API and features have changed significantly since then. Often there is no post describing the changes. Even if there is, it now takes twice as much work to read both posts and synthesize an understanding of the project.
Learn the Lessons of Your Predecessors
Typically there have been projects like yours before in your technology stack or in others. They have tried what you are doing and discovered the pitfalls and best practices. Look carefully at other projects doing what your project does. Make sure you understand how they work and why they work that way. Read through the issues they had. You might discover your exact approach has been tried but found wanting. You will likely save yourself a lot of time in the less core areas of the project which you haven’t been carefully thinking about.
A Slick Website Looks Professional
As developers we don’t value beautiful marketing material as much as we should. Even though your project isn’t a product you are selling for money, you are basically selling the use of it to your users. Having a slick website to sell the project gives people a good feeling about your project and makes it look very professional. If you can invest the time for a great website, then you must have the time to invest in a great project. The site doesn’t have to be large. Even a couple pages of description, features and contact information with links to documentation, code and downloads will go a long way.
Make Sure the Project Runs out of the box
If you have a sample project or example of using your library, be absolutely certain that it works immediately out of the box after users download and install it. If a user’s first experience with your project is that you can’t get the sample to work, they will be left with a very bad taste in their mouth. With each new version, test the sample. If possible make unit tests to cover as much of it as possible. Another way to fail in this area is to have the download instructions or link not updated so that they point at an old or broken version.
Practising What I Preach
Looking at all these open-source projects and realizing the important things I listed above, I realized some of my projects are failing in one or more of these areas. My next step is to go through each of the projects I have posted online and address these.
In summary, here again is my advice. The first two are for all open-source code posted online. The rest are for serious open-source projects:
- Clearly Indicate the Status of the Project
- State a Clear Value Proposition
- Recognize the Commitment and Make it
- Don’t try to be All Things to All People
- Docs Are as Important as Features
- Learn the Lessons of Your Predecessors
- A Slick Website Looks Professional
- Make Sure the Project Runs out of the box