As I look to reduce the number of touch points that are required to make a change to my personal system, I’m always thinking about how to reduce complexity and leverage existing tooling. In this way, we can continue to leverage an existing foundation to build out further abstractions. One such lever I have worked on over the last year was using my gRPC protobuf definitions to generate LDAP schema files that could be loaded into the server to match the objects that I’d be working with in Go.

When I initially stated implementing gRPC in my environment, I hadn’t expected to be developing my own code generation methods, even though the gRPC and protobuf definitions support this by default by creating language-native methods for the developer to implement.

To do this, I’m using an interesting project by Moul on github. This project reads the proto file and then injects an object into any Go template that can be used to generate pretty much anything based on the contents of a proto definition, which is super handy.

As I worked on my system, I realized that many of the objects I was trying to work with in Go looked a lot like the objects I wanted to store in my database. In my case, the database I’m using is LDAP, but this method applies more generally than LDAP in particular.

After splitting a monolith proto file into many diferent packages, I then updated the make targets to ensure that each proto file also called the template plugin to generate the text output I desired.

proto-grpc:
	@protoc -I internal/inventory/ -I ./ \
		--gotemplate_out=template_dir=internal/inventory/templates,debug=false,single-package-mode=true,all=true:internal/inventory \
		internal/inventory/inventory.proto
	@protoc -I internal/inventory/ -I ./ \
		--gotemplate_out=template_dir=cmd/templates,debug=false,single-package-mode=true,all=true:cmd \
		internal/inventory/inventory.proto

The above is a little hard to look at, and took a few minutes to get straight, but now works pretty reliably for my needs.

The next part of the project was to start developing the templates themselves. I started with the LDAP ldif file. I’d worked with templates in other languages and a little Go templates before this, but I still had to work at getting this right, which took some time. You can see the current template here and the resulting ldif file here.

Once the schema is generated, the last piece I’ve not yet automated here is just to modify the database schema itself.

ldapmodify -Z -H ldaps://auth10.znet -D cn=config -W -f internal/inventory/ldap.ldif -vv

Next I wanted some methods that I could use to interact with the database directly, without needing to implement that client side. This would methods like Create, Read, Update, List for the various objects I wanted to implement. This code ends up being pretty wordy in the template and the resulting output, so I’ll just link out to it. Here you can see the template, and the resulting code that was generated.

With the above in place, I’ve been able to scale out the system from just a couple of objects, to several dozen at this point. I have an easy way to update the database schema with objects from the protobuf definition, and Go methods to interact with those database objects. This has come in quite handy, and even though the templates too a few iterations to get right, I’ve been happy with the results. Now all it takes to add a new field, or object is only a couple of lines worth of manual change, and then a make proto at the root of the project and everything gets updated. I’m sure there will be changes to this over time, so the inventory package should be the place to look for the bigger picture.

I may end up moving this sort of approach to Cuelang in the future when they support service definitions from proto, but for now I expect this to carry me through the year.