Practice English Speaking&Listening with: Grindr Tech Talk: iOS Development with Test Driven Development

Joel >> I’m Joel Simkhai, I’m the CEO, founder of Grindr. So I want to welcome

you guys to our talk tonight. Before I introduce Lukas, I just want to give you a little bit

of a background on Grindr. We’ve just celebrated five years a few months ago and we’ve started here in Los Angeles.

Actually I was telling someone over in my living room for about two years and now we’ve been here in this office for

about a year now; a little less than that, and we’re global. We have about 5 million;

over, actually, over 5 million monthly users who go on at least once a month and on a daily

basis we have about a million and a half who use the service every day. Concurrent, we’re

about three hundred thousand, two hundred and fifty thousand users who on at any given

minute so a lot of traffic, a lot of usage, a lot of guys grinding, and this is global,

so this is not just here in the US, this is literally any country you could name, we’ve

got users there. And, this is, you know, it has to mean to you the power of the service.

Helping guys meet other guys seems to be very popular and we love to do that for them.

And the other kind of interesting fact is that we’re self funded, so we have

two versions of the app. We have a free version, and a subscription service. The free version

is ad supported and the subscription service is a monthly fee and so between those two

revenue streams we support the business here and you know, so we’ve never actually taken

any outside capital. And something else we’re very, very proud of is that we’re committed

to just creating great products and just doing a great thing for our users and for the team

here and not really worrying about investors and spending time on pitches and all these

other things. We’re focused on creating great products. So with that quick background,

I’d love to introduce Lukas Sliwka who is our VP of Engineering and the Head of Tech.

Lukas >> And I broke the microphone. Ok perfect. Welcome everybody. Today we’re

going to take a deeper dive into iOS development, Xcode specifically. We’re going to focus

on a couple of areas that are very important to me. I joined Grindr six months ago.

Grindr experienced explosive growth as a company, and with that growth came a lot of development

in terms of backend systems, the frontend clients. And when you’re in that mode when

you’re growing so fast, a lot of times, you’re kind of throwing things together

to keep up with other men, right? And it shows. It shows on our backend systems, it shows

on our frontend clients. The first talk I did about

two months ago, I talked about scalability, and so, during that talk I talked about what

it actually takes to support millions and millions of users, and that’s not an easy thing.

Today we’re going to focus primarily on iOS side of things, and how you can actually

employ some of the strategies to best develop for your iOS clients.

So before we go on, I want to introduce a couple of people who are going

to be helping me along with this conversation. So, I’d to introduce John King who is our

iOS developer and part of our iOS development team and also Rudi Hacopian who is our QA automation architect.

Before I go forward, I want to, kind of set the stage in terms of, you know, a lot of startups, a lot of applications

being built, you have maybe one guy, two guys, working on in a backroom, throwing things

together, right? That’s all great as long as you know frontend and backend and all the

ins and outs of scaling your application. There is a big difference between handling

a hundred users versus ten thousand users versus a hundred thousand users and the scale

goes on. I mean, as your organization grows, you need to have certain specialties, right?

However, when you look at startups, you need to stay agile to help people who are kind

of jacks of all trades but at the same time they are masters of few areas. So when you’re

trying to set up a very successful startup or a company, a mid-sized company that is

very lean, you can do a lot with 15-20 people as long as these people are very passionate

about technology and you build a framework around them that allows them to innovate,

fail fast, learn, rinse and repeat. So we’ll talk about some of that today.

So let’s start by discussing, you know, what we’re going to talk about

today, so, specifically when it comes to iOS development, we’re going to talk about Test-driven

Development (TDD), we’re going to talk about Inversion of Control. Another name of that

is Dependency Injection. We specifically use Typhoon container for that. We’re also going

to discuss Dependency Management powered by Cocoapods. We’ll talk about Burn, Build

and Release Pipeline and we’ll talk about what makes a mature build and release pipeline

for a mature organization. I will like to set a low-context why I want to talk about

these things, alright? When you’re a company and you’re a startup, you want to have an

environment that enables innovation. My job here is to essentially build and engineer

an organization that promotes rapid change and rewards experimentation and that’s not

an easy thing. As a developer, and if I could get a head count – how many developers in

the room, people who actually write code? So most of you. So you know what I’m talking

about; how difficult it to be in organization that has production applications already supporting

revenue and that’s locked in and at the same time you’re trying to deliver all new

projects and at the same time technology is moving so fast, you want to experiment with

different technologies, possibly apply new things to improve things. It’s a constant

battle between having a stable production environment that generates revenue, have a

development engineering environment where you can innovate and you can agree with the

product because product development also wants to deliver features so it’s a constant collaboration

and a constant negotiation between those different forces.

So what I want to talk about today is what it means to have a strong engineering

practice. When I started with Grindr six months ago, my main charter was to build a strong

engineering practice, and what I mean by that is, build a mature engineering practice that

essentially enables people and allows people to experiment, allows teams to experiment

and possibly fail fast, learn without affecting revenue, without affecting these core business

applications that are already running. Joel mentioned that we are

a mature business, that we’re making money. We have a lot of users in over 190 countries

around the world. We’re processing massive traffic. Obviously, we’re not Facebook,

we’re not Yahoo or whatever, but you know, we deal with close to 200-500 messages per

second when it comes to chat. We talk about traffic like 250,000 RPMs to 500,000 RPMs.

We have days when we reach one million RPMs. Those are not easy things to withstand.

When you’re designing your backend systems and also frontend clients then there will soon

be a lot of thought put in on how you orchestrate things. When it comes to the setting up of

a strong engineering practice, however, there are a couple of different components that

are a part of that and a lot of the stuff we’re going to talk about today are the

different building blocks of the overall picture. I just want to give you a context of why we’re doing it.

Why are we doing TDD, or why

are we using IOC container or why are we writing Unit Tests? It’s not because it’s something

cool and someone else is doing it. They’re doing it but there’s a purpose behind it

and the purpose is to have that strong engineering practice. A strong engineering practice is

composed of three components, which are, processes, technology and people. I kind of touched already

on people and who we hire, who we want to be part of our team. Our engineering team

is very small, we have right now, probably fifteen to twenty people that are making this

whole thing happen. They are divided into various different functional areas so we have

standard dev ups, NOC type of practice, we have QA practice, we have architecture practice,

we have android, iOS development, we have backend practice. All these kinds of practices

have to come together in some way to create a product. We use agile and scrum for that.

I don’t believe that any sort of modern software development should be using anything different.

I think that waterfall is dead. I am not afraid to say that. A lot of companies

are claiming that they’re agile and scrum. In my opinion, if you have a status meeting

in the morning that you call ‘standup’, that doesn’t make you an agile organization.

There’s much more to that, there’s going to be a separate tech talk on that where I’m

going to take a deeper dive. But there are also other things, like Lean and Test-driven

Development and there’s KISS, which is, ‘keep it simple, stupid’ or YAGNI, which

is, ‘you’re not gonna need it’ or metrics. It all comes down to the fact that we want

to move very fast, we want to have a process in place that kind of sets a cadence to day-to-day

development but, as an engineering organization, we also have architectural principles in place

that are being governed and watched for in terms of how we build software.

In terms of technology, I am in a certain business, right? So I am enabling

the gay community to meet as fast as possible. I’m not in the identity management business,

I’m not in the business of building databases, I’m not in the business of other engineering

aspects that are very cool for engineers to play with, however, that’s not my core business.

So it’s very easy as an engineer to get caught up in the rabbit holes where – “oh

you know what, I’m going to build my own stuff, right, because it’s cool.” That’s

not how we do things here. If we’re looking at the overall ecosystem of the Grindr application

as a whole, I want to channel the development effort into areas that haven’t been done

before and maybe there are no appliances or frameworks on the market to do them. I want

to make sure that I apply open source technologies, I buy whatever I need to buy, and I integrate

to build a strong solid platform that enables my team to do things that haven’t been done

before. I don’t want to be in a business building cool geospatial algorithms

because there are plenty of companies that specialize in that. The databases that have open source

algorithms integrated inside them like Mongodb uses Google geospatial search.

I want to leverage that. I don’t want to build that from scratch. When I joined Grindr, I looked

at the overall ecosystem. There’s a lot of things that we’re fixing because, over

the years, because we’re moving so fast, it was easier to build things from scratch

as opposed to maybe take a step backward, look at our architecture and design things

differently, so this year what we’re doing is we’re redesigning backend and we’re

completely revamping our system, the overall platform as well as building clients.

[13:50] Lukas >> When it comes to clients, it’s kind of the same thing, right? When

I look at Xcode and iOS development, iOS development just recently, last year, introduced the version 7

which brought, finally, unit testing tools and the ability to do things that haven’t

been around for a very long time. It’s amazing to me, a lot of times when I talk to iOS developers

- I don’t want to offend anybody - how special iOS and Xcode development is,

when I tell you that you’re not special and the reason why you’re not special is because software

engineering has been around for a very long time, and the concepts that have been introduced

into your ecosystem have been around for a very long time, and there is a reason for

that. The reason for that is because for a very long time we’ve done object oriented

programming and we’ve solved things through frameworks and microframeworks, and patterns

and those things should be applied. Xcode and iOS development is not different from

Java or any other type of language, so what I want to stress is that things that we’re

going to be talking about today are maybe new things that were introduced quite recently

into the iOS development ecosystem, but they have a very strong foundation in computer

science and the backing of at least 10 to 15, 20 years of object oriented development.

So, when I talk about frameworks, containers, automation, that’s something

that we use here to essentially provide these tools, provide the technology, and the frameworks

and the process to allow our teams to experiment, learn, fail fast and do it better the next

time. We work in one week sprint cycles so that means that within five days, you need

to complete your development, you need to confront your definition of done which means

that regression testing needs to be done. You need to have a certain level of quality.

You cannot achieve that without automation and a certain level of maturity as an engineering shop.

So, let’s talk about – before we take the deeper dive, I wanted to throw some definitions out there for people who

have not done before. Test Driven development, have no doubts with dependency management,

don’t know what the inversion of control is. Essentially, those are tools in our toolbag

that we can apply to make our development faster, better more robust and at the end

of the day we allow people to experiment with different framework, different technologies,

different approaches, without worrying if we’re going to put something in production

and that’s going to bring me down and my revenue is going to stop. I want to have a

process in place as an engineering shop where I enable you guys to experiment, but in order

to do that I need some of these tools. [17:05] Lukas >> I’m not going to go through

that, you guys can look it up on wiki. What I do want to do before we go into the code dive is

I want to take a brief moment to talk about inversion of control and testing.

So I set up essentially a very simple scenario, where I have a Class, ‘Grindr’,

that uses Grindr service and normally, if you don’t have any serve containers

around that stuff you would just say, you know, I would instantiate my new Grindr class

and if I’m referencing this Grindr service, I would have to have either in my constructor

or right here, some sort of way that I’m instantiating this class or I can do it within

my business logic so it’s populated. When it comes to Inversion

of Control, it’s essentially a container that allows you to outsource, for the lack

of a better word, the business of creation and wiring classes together.

If I have two classes like this, my IOC container is essentially a way for me to create this class through

that container without me worrying about how that’s done. Why it’s important, well

if I’m doing test driven development, I want to have very atomic tests that are just

testing this particular component, which means I will have to, most likely, mock this service

and mock the responses of that service in order to have a valid test.

Very high level, I’m not going to talk about more than that. I want to hand it off to John, who’s going

to actually look at some code, show you how it’s done in Xcode, starting with Dependency Management using Cocoapods.

John >> Hey everybody. Luke was kind enough to introduce me, I’m John King. I am an iOS developer here at Grindr.

Who here has not heard any of these concepts before tonight?

Fair enough. Alright, so the first thing I want to talk about is Cocoapods.

Oh! You can see what I’m talking about, huh?

Awesome. So Cocoapods is what we use for dependency management for our Grindr project.

Cocoapods is in itself simply a Ruby gem we installed through terminal which will

keep a reference to all the various key repositories that are managed by Cocoapods and gives us

a really easy way to put them into our project.

To simply install it, we open our terminal and we gem-install Cocoapods just like you would in any Ruby gem.

Simple enough and if we go into Xcode we can look at the key part of a Cocoapod. So this is

called a pod file. A pod file is how we tell Cocoapods what dependencies we need for our project.

This can be opened up as any simple text file and that’s exactly what this is.

That’s a text file with some glorified Ruby in it.

Let’s go ahead and break this down for a bit. So you can see we have

a couple different build targets, if you’re familiar with Xcode, that our entire workspace we build,

for each target we tell them what particular pods we need in order to build

this project for the winter. A pod is just a Cocoapod term for a framework that we use

within this project. So you see we have four various

targets, they use all very similar things. Next to them you see a version number.

This is an optional version number. If you omit the version number it will always install

the latest version. At any time you can run a different command called a ‘pod update’

and it will take every dependency you’re using and automatically update.

We don’t have to worry about - oh has an update come out yet, have we received

an email about it, is it up for release? Let’s go check and see if we have the latest versions

for everything that we’re using. We simply use a pod update.

For certain reasons we have frozen every particular pod we’re using. We’re telling them this

is the version we want to use. Always go and get this version. Cocoapods uses a file called

a pod spec to give it all this information about how it might build in with your project.

For 90% of new developers this is something you’ll never have to work on unless you’re

building a pod for someone else to use. Pod specs are there, you need to know they exist.

This pod file is the more important thing. There is a particular command that you see

here. Looks like a tilde arrow. We can actually do something like this if we so desired and

this would give us the latest men version of that file or of that pod, forgive me. So

let’s say I’m using Yama framework and they have a new – I’m sorry, go ahead…

Q: [Inaudible] John >> Oh yeah, I’m sorry.

That’ll work. Alright, how’s that guys?

Apple says this is a good presentation size, it’s good enough for me. Alright, so, sorry, should have said something sooner guys.

So you see we have a tilde arrow here, we replace this with just our comma so instead of using the version we’ve given it,

it’ll use the latest men version which is the second number here available

to us. So let’s say Yama framework as you see here are – we currently use 0.2 of it.

Let’s say they have a new 1.0 release of it but there’s also an 0.3, 0.1.0, it will

give us the latest second number that is available without moving to the next incremental. This

is useful if we want to keep using a similar version that we have but we are weary of using

the latest and greatest in case there are any bugs, in case it doesn’t play right

with our archives for any reason. It’s a handy little tool if you’re using your little pod file.

So, this is the second most important part of the pod file for me. So we got a post install hook and this is our

chance to write and review code and do some things to help with integration with any of

these pods for example, we have a number of Xcode schemes that have their own particular

settings to help us build and maintain the project and often times just doing a pod install

- very basic - will not transfer these settings over to that, does not play nice. As I said

earlier, a lot of what we need in order to make these libraries build properly with us is in that pod spec file that Cocoapods uses.

Now, what I’ve done here is a too much feinty Ruby Regex. All it does is take all those settings we need, copying

them into what we’ll be using and tells us it’s done. Simple enough but you may

have any other use for that post install hook is catchall for we have downloaded, we have

installed the pods, we have integrated them into the project, do you need to do anything else?

It’s important to know after you run a pod install, it will generate something called an Xcode workspace. For most people,

if you create a new Xcode project, it will be using something an ‘xcproj’, an Xcode

project file. Similar to a vcproject in Visual Studio, but we need to use multiple projects

for this and what Cocoapods will do is automatically generate an Xcode workspace with everything

we need for it and that is the file we use from now on. You can forget about that Xcode project for time immemorial.

Before we move on does anyone want to ask any questions about Cocoapods, why you might use it, how it might be used?

Q: [Inaudible]

Excellent question. So Cocoapods has an awesome website, Cocoapods.org. You’d just like to find any other open framework.

It can be difficult to just know it’s out there and then once you find that it’s out there, what’s a good one to use?

So I can go into Cocoapods.org and I can type in a generality of what I might be looking for when I type in ‘jasonkit’ here

and it tells me what pods are available for that and all I have to do is plug in what I find here into my pod file.

Q: [Inaudible]

How I pick one? That’s kind of up to the individual developer.

It’s kind of like asking, ‘how do I determine what framework to use?’ Sometimes that just

comes with the individual’s knowledge, sometimes it comes with trial and error. It’s kind of a personal choice.

Lukas >> What I would suggest in those cases is that you do some sort of POCs, so if you’re evaluating let’s say three or

four different frameworks, what we tend to do is we come up before with an evaluation criteria,

so if you have ten things that you are looking for, you use a simple Excel spreadsheet.

On the left hand side you got your criteria. You list all your frameworks at the top and

then you can actually write little POCs using each and test the performance of whatever

and you can have actual metrics behind which one you wanna use.

Another thing you have to also look at is the [illegible] project how active it is.

How many people you have committing to it. So you have whatever, Github or whatever

repository you use it’s good to check how actively they are updating it, right?

Any other questions?

Q: [Inaudible]

Which parameters are we talking about?

Audience: [Inaudible]

Correct. Yeah, these are hardcoded version numbers saying that these are the version numbers we want to use. Do not use

anything later. Do not use anything older. Use only this version.

You have a choice to actually select something that will automatically load

the latest version that is better than this because, you know, what if they deprecated

some stuff and all of a sudden you do a build and you’re stuff breaks. So, you need to

a entire ceremony when it comes to dependency libraries, when it comes to updating and it’s

usually a project that you want to schedule like, ‘ok there’s a newer library for

this, let’s actually have a user story in your agile port for that so you can actually

refresh it and test it for some stuff.

Very cool. Anybody else before we move on?

Awesome. So the next thing I would like to talk to you guys about is the Typhoon framework. The Typhoon framework

is the dependency injection or IOC container we use within Grindr. There are a couple of

IOC containers out there. You can find some small projects on Github but a lot of them

are either not being supported any more or have very little adoption but Typhoon seems to have risen to the top.

If you look online for any number of times for an IOC container for objective C, you’ll see Typhoon’s name was said

a number of times and it can be found at typhoonframework.org and from their website you can go their Github page.

There is also a Cocoapods for Typhoon in order to integrate it in.

Let’s talk about some of the base things of using Typhoon.

The base part about going into Typhoon is you have these logical groupings called, ‘Assemblies’.

Assemblies are a group of related instance classes that can be used.

These can be later fed into our creation factory and they are how we will access them later

So you see I’ve created a class here called ‘inpoint log assembly’ and it’s derived from Typhoon assembly.

All assemblies are derived from Typhoon assembly and we return to type id and we have instances we will need to retrieve.various

Within our involuntation file – wow this is big.

We return, we tell Typhoon, we give our class that what we need and then we have a chance to do some things to it before it’s returned.

So, for example, for the inpoint of log we’re telling it, ‘hey, this is the class we need and we need to inject it

with this property which is also shared within the assembly which is our inpoint of log entry

factory and we’re also setting the scope. Setting the scope is a really

important thing and it, kind of, can be difficult to understand for some things.

So there are four major types of scopes.

The first one is TyphoonScopePrototype, which, anytime we request this object, we

will get a brand new instance of it, everytime. No reviews.

The second and the default is the type object scope graph. If you do

not specify the scope, this is what it will do by default.

Object scope graph, object graph scope, rather, forgive me, is an interesting choice and I’m a big fan of it.

Let’s say I’m in the view controller and I have some sort of delegate and there’s an object we need to pass between them.

Let’s say within our code base there’s also another view controller and delegate and similar object so we don’t

really have a singleton that’s passed between but we won’t have to try and pass the object

back and forth. With Typhoon object scope graph,

any time we request that object, we will receive the same one within a similar

frame of execution so let’s say I have requested this view controller from Typhoon for whatever

reason and I am working within it doing some things and I request something else within

it, it will return me a new instance of that object.

I’ve now gone somewhere else within this execution stack, and I request

that object again, it’s going to return me the same object, no different, the same

one. Now, I’ve gone out of it and I’ve gone into a new view controller to do something else.

It will get a new instance and its delegate will get the same instance of that.

It’s very handy, very, very handy. The third type of scope you see here

on the board is TyphoonScopeSingleton. Does everyone know what a singleton is? One and only one. Awesome.

There’s also a fourth type called weak singleton. A weak singleton is just like a weak property that zeroes out if nobody has a reference to it.

Awesome. So let’s talk about how these assemblies are used. Does anyone have any questions about the assembly before we move forward? Very cool.

Lukas >> Gimme a while, I’ll change your resolution on the screen.

John >> I’m not very good with… can I use built in retina display?

Lukas >> That should be a little bit better.

John >> It’ll work. Alright, so you see here. I’m initializing a Typhoon

component factory and I’m feeding it a number of assemblies. You see, I have… uh, it’s

a little difficult to read here, forgive me. This is an array of various

assemblies we have within the project. And these are all the different assemblies it will manage.

Now’s a good time. Uh, let’s see.

Bear with me for just one moment.

So, the question comes up, ‘how do we access this factory at runtime,

when we want to retrieve things from it?’ So there is a protocol we

can use for our classes called TyphoonComponentFactoryAware. Whenever any of our instances get initialized

through Typhoon and we are TyphoonComponentFactoryAware, it will call setFactory on that object and

we now have a reference to whatever factory we are associated with and are gonna retrieve

anything from those assemblies, anything from our own assembly.

And it’s fantastic because all we have to do is cast the factory as an assembly and call the method as normal.

If we are not TyphoonComponentFactoryAware,

there is a convenience method called makeDefault and we now have a default factory that we

can access anywhere else. This is mainly used in legacy code.

Now I will touch on Typhoon again when we start talking about Unit Testing.

Does anyone have any questions about basic use of Typhoon? Very cool.

Alright, so who here has done Unit Testing in iOS?

Alright. Tell me, how easy did you find it to use?

Audience >> Just my first time so… John >> Fair enough. How about you, sir?

How easy did you find Unit Testing to be done in Xcode?

Audience >> It sucks. John >> Little bit, yeah. Little bit.

So in order to do any Unit testing in iOS, we create an instance of a class called xctestcase. We derive from it,

we name it. We got a new class. Unlike other classes that

Xcode will create for you, we only have an implementation file. You can have a header

file but there is typically no need for it. There are one, two, three,

four, five major methods to know in any Unit Test environment – are in any Unit Test,

I should say. We have a class level setup.

And this is a chance to do any type of initialization before any tests

are run at the class level. This is run only once before all testing.

We then have an instance level setup which we run before any test is run

so this is our chance to setup any variables, get the environment a certain way and it’ll

be run once for every test case and then it will run any method that begins with the word,

lower case ‘test’. Objective C is definitely

coating my convention and this is a good example of that.

It will go through any method that begins with the word, ‘test’, no

matter how it ends in range of those. And then after each test it

will run an instance level teardown which is our chance to clean up after ourselves

and after all tests are run, we have another class level tear down.

Which is a chance to clean up anything we’ve left over before the test has a hole.

In order to run our Unit Test, we simply hit Cmd + U. It will open up the simulator for you.

I’ll begin writing our tests. You can see them pass in realtime if you have

your pick of the code file open. And you’ll see at the end that all our tests have succeeded.

If we see a failing test…

It will tell us overall that all our tests have failed and it will give us where it is and whatever particular error message we have printed out.

So we determine whether or not the test has failed through various Xcode asserts

And we can see we have a number of different assertions.

Let’s say you’re brand new to Unit Testing, you don’t know any

of these, this is all you have to do, actually the assert. This is just like a standard NSAssert.

Any false value will throw an exception and will have failed the test.

One thing that’s incredibly important for at least our unit tests and

arguably all unit tests is the ability to mock objects.

In any of our testing, we try to keep… we only want to be unit testing

the class our tests are built for. We’re not doing integration testing, we’re not

testing how it talks to any other class or even our backend, so we use what’s called, ‘mock objects’.

Are most people familiar with that term, mock objects? Very cool.

So, one deficiency that the Xcode testing suite has is it does not include any type of mock library. There’s no ability to create a mock object natively.

There are a number of libraries useful. The one we use in-house is called OCMock. Once again, this is another really

popular library. You’ve seen lots of wide previews of this.

There are a couple… I’d like to touch on just the highlights of mock,

some of the important methods that you would use for this.

So there are two different type of mock objects that you would use. One

is a more traditional live object where it will only expect methods that you have told

it to expect and if it receives anything else it throws an exception and it fails the test.

We have another type called a niceMock and we can, sidenote – we can

mock those classes and protocols. A niceMock of a class or protocol

will simply accept anything you throw at it but not return anything for those values,

it will simply return NIL. So, we also have the method

called Expect on mock objects and the Expect method gives us… we tell this mock object,

‘hey I am expecting this method to be called when I later call another method’, which

I’ll go over next, ‘if you did not receive this message, then you need to throw an exception

an fail the test.’ That the method we call at

the end of our test in order to determine that all expected methods have been called

is called, Verify. Let’s say we simply need

to stub a method, there is a handy-dandy method called, Stub. We feed it what method we expect

and we can also give it a return value in order for it to work nicely with anything

else we might be testing. The last thing I wanted to

talk about in terms of Unit Testing is I wanted to touch on Typhoon again, and I wanted to

talk about the TyphoonPatcher class.

If you’ll give me just one moment…

So, the great thing about the Typhoon factory and the assemblies is, we set everything up and we go to runtime

and it works as expected. Let’s say I’m writing you a test and the class I’m unit

testing uses some things from the Typhoon factory but we do not want something from

the production assembly. This is a very common case. We don’t want to drag in any other

classes that we’re not crisscrossing for unit testing in this particular test.

In order to do that, we create our factory with whatever assemblies we need

just as we normally would and then we create an instance of TyphoonPatcher. What TyphoonPatcher

does is that it gives us at runtime the ability to modify what the factory will return.

So you see what we’ve done here is, for the factory that we’ve created

here, which includes just the inpoint log assembly. We patch in for the inpoint assembly,

or for the inpoint log assembly factory, forgive me, method, we create a mock in point log

entry factory told to expect and return some things and return that, and in order to give

that information to the factory, we then attach it as a post processor and from that point

forward, that factory will return our mock object that we expect.

Alright, and that I believe is all the code examples I have for unit testing Cocoapods in typhoon. Who wants… sure.

Q: So within the unit testing,do you have like, development, QIs and production environment or what’s the...

Our unit tests are run on… Rudy can talk about those next on how unit

tests are run as as part of our continuation integration environment, although we do run

them locally before we commit. Luke >> So the idea of unit tests

is that, that’s a suite that I’m expecting every developer to execute when they are making

code changes in their local environment but it’s also part of our continuous integration

environment which will talk about the build pipeline.

So whatever you checked into repository then that is checked out in continuous

integration process and those unit tests are run again.

John >> Certainly.

Audience >> When testing [can’t decipher] can you run it outside of a simulator?

John >> This simulator runs every time. It doesn’t run on Vice, it does not run outside of anything else. This simulator

will open every time. You don’t see anything, the active self doesn’t run, it’s only run in unit tests.

Audience >> [can’t decipher]

John >> Keep in mind the Aphen simulator is not an emulator. You won’t get the speed

of the device you’re using, but no, it won’t run from the Mac OS.

One caveat to that, unless you’re not doing an iOS app and you’re

doing a Cocoa app, an actual Mac application, then they’ll run in its native environment.

Audience >> Can you verify that certain values will pass to the given function?

John >> Yes, through mock objects specifically, when we are expecting a particular

method as argument for whatever method we’re expecting. We can give it either the value

that we expected to get, and if it does not receive that value, the test will fail, or

we can give it an instance of a class called, OCMArg, which you can do some fancy things

with it to determine what it passed in, and you know, ‘does this pass certain variables’

or we can just take anything. Whatever passes, we don’t really care, just we have to have

something to be able to accept it.

Audience >> What if we need to call the method twice? How do we specify that?

John >> You simply expect it twice and everytime you call and expect on it, it’s going to add another… it’s going to increment

another counter and everytime it’s called it’s going to take one off the stack and

then take it off the stack again if we’ve expected it twice and if it receives another

one, then it fails because it was not expecting it that many times.

Luke >> Cool. That thing we talked about Cocoapods, very important concept. Again,

Cocoapods allows us to manage dependencies you saw a lot of libraries use. OCMock for

example, so you want ot have an ecosystem that is managed through a framework.

How we low these things by pulling framework again allows us to essentially delegate the factory

functionality to a container which then is a byproduct of that allows us to essentially

write unit tests and essentially isolate classes and write very specific test for those classes.

If there are any dependencies between those classes, Typhoon can essentially patch those

classes as OCMock objects. So we essentially can write very expected returns or very expected

results for execution so our tests are very atomic and that’s very important. You don’t

want to be testing fifty thousand dependencies as one, you want to unit test your one class.

It’s very important distinction to make between unit testing and functional testing.

Functional testing allows you to test a lot of different things. You’re focusing on

business logic and you can use automation for that, and that’s a different topic,

So let me talk about built process and how it all kind of comes together with a delivery to production and I’m going

to start by showing you where we were six months ago, and let me click on that. So…

oh that’s not cool. Let me do that… and let me do that. Can you guys see that or no? Ok let’s do this.

So, this is essentially a diagram of how our release management process looked like. Let me very quickly focus on

couple of key areas here. So if you’ve done iOS development, you know the concept of provisioning

profiles, certificates. There are various different types of enterprise ad hoc productions

of easy development. So the way this used to work

is that, we actually had our iOS developers maintaining all these certificates through

iTunes Connect and then, they would do their development of the iOS machines and essentially… stop it.

And they would essentially do their local testing by checking out these certificates

that we would export to our Bitbucket repository to do their local essential build which was,

you know we would run Xcode build, it would create Grindr.app, they would then sign it

with the development provisioning profiles and their certificates and then, now we are

able to test our device on the local machine, and they would check in that code into Bitbucket

and then we would our Jenkins process to check that code out and we go through the same process

again where we kind do an Xcode build, we create Grindr.app. We sign it with any of

those. We would have a publishing process that would create these IPAs for enterprise,

Ad Hoc, and when we would use Hockeyapp to… we’re using YouTest for crowd-sourcing our

tests and also our internal QA would then use the Ad Hoc builds to test, right?

Now, where it gets really funny is that now, ok I’ve done all my testing,

it’s great and I’m ready to go into production. And then what happens, iOS developers check

out the code to their local box and they go through this build process and on their local

box they’re signing the IPA, I mean, APP with production of provisioning profiles and

then we would upload to iTunes and then eventually it would get to the App store.

Now, when we talk about mature, you know, build management processes, there

a lot of problems here. Number one, developers have access to production and certificates

and provisioning profiling, allowing them to essentially deploy stuff to production.

That’s not what I necessarily want, right? Another thing is, what I’m

testing here is really not what I’m deploying. There is no release candidate. Because, ok,

I’m gonna go through all this effort, and now if my developer foobars their check out,

or if someone between me doing testing, goes in here and checks in some additional code,

then my check out will catch that and in fact, we had a couple of instances where we put

stuff to production that we really didn’t want, right?

Amazingly so, this is probably majority of iOS development shops do it that

way, and the reason why is because for whatever reason the tools are not there, the environment

is not mature to automate this process, so we actually spend a lot of time and effort

to build a release management pipeline to do it more like this, which is our new release

process, which is, we have obviously these provisioning profiles, we have release management

function that maintains these. There is a separate Bitbucket repository that governs

these things with separate permissions so developers do not have access to that stuff.

Developers can only do development on their iOS machines. They do this process, they use

the development provision profiles to do unit testing and local testing, but once we get

to the Jenkins box, Jenkins box actually checks out the code, checks out the provisioning

profiles. Thus, through the build process it creates the versioned release candidate

that is then checked into Nexus artifact repository before the signing process so that’s my

release candidate. My APP is now a release candidate which I then can draw from and publish

different environments. Once I certify then I use the same artifact from Nexus artifact

repository. I’m going to check it out, sign it with production, provisioning profile and

push to an App store and that is being done by my release management not my developers

and that’s more in line with the mature enterprise build pipeline release management

process. Quentions. [54:00] Audience >> Do they have to access

into our iTunes’ account or two different [inaudible].

[54:07] Luke >> Essentially developers don’t have access to… yeah… ok?

[54:14] Luke >> So that’s really what you want. Key here is the release candidate and

actually the usage of the artifact repository. Another thing here, Sonar will talk about

code metrics. I’m very big about having empirical data about what my code is like,

you know? It’s not a what a cool conversation, I want to see actual metrics, they’re standard

metrics to measure the quality of my code. We’ll see that in a little bit. And also,

I want to have automated testing as part of the process. So this build, everytime I do

build, right? There is going to be unit testing. I’m gonna calculate my metrics. I’m gonna

also run my automated fuctional testing, and that’s the future we’re working on that

right now. We are doing a big push to have that done by the end of the summer. At the

end of the day, everytime I do a build, I want to have a couple things get done. I want

to calculate my code quality metrics and I want to fail the build if someone wrote something

without the unit test, which means my code coverage percentage decreased. Also, you know,

if my unit tests fail, obviously I want to fail the build and once on top of that once

the artifact is done, I want to run my automated testing using functional tests, which means

that now I’m actually testing the out functionality, right? And that should happen every night,

at least. Right? [55:42] Luke >> So now, developers can innovate,

they can do whatever, they can test different frameworks. There’s a harness in place to

fail fast, preventing me from pushing crap to production.

[55:54] Luke >> So let’s talk about how it’s actually implemented, so Rudy, if you

could take over and talk about our Jenkins setup and the pipeline.

[56:21] Rudy >> Alright, hi guys, so I’m now going through some of the stuff that Lukas

and John went through in a little more detail about how the builds are happening, how the

artifacts are stored, how do we track them, and what we do with them after our QA signs

off on them, right? [56:36] Rudy >> So Lukas briefly mentioned,

now what we do is we basically store we build. So I’m gonna start with a couple of tools

we’re actually using in-house. So we’re using Jenkins for all the build processes.

So we use Jenkins to build the artifacts, we use Sonar, if you guys are familiar with

it, for reporting. So what we do is after we build, we run the Code Quantmetrics and

report it to sonar. So I’m gonna put up Sonar and show you guys the dashboard, it

basically tells you where the violations are, what code is missing, unit test coverage,

if they added five classes, and they haven’t unit tested obviously, the coverage percentage

drops so we can flag it as a failure and ship it back to the dev team before it even makes

it to QA. [57:17] Rudy >> We’re using Nexus, it’s

an artifact group repository. So what we do is, after we have the APP and the IPA artifacts,

we store them there for tracking purposes with the appropriate Git hashes so we can

trace it back to which Git commits they were associated with and we can troubleshoot them

if we need to. [57:35] Rudy >> And Hockeyapp obviously wants

to, if you guys aren’t familiar with it, to doing distribution of the IPAs.

[57:43] Rudy >> I’m gonna start with the iOS sold with the Jenkins piece. So what we

do is we basically use Jenkins to build the artifacts. What we have done is our new build

process basically is the… we’re tying down a couple moving parts to together to

make this happen so first we start off with Code Quantmetrics. Do they pass? If it does

pass the code quality and we’re good with that, they have done proper unit testing,

all unit tests pass, they haven’t broken anything existing. We go ahead and build the

app. Once we build the app, all this stuff is still done in Jenkins. Jenkins will build

the app after doing the reporting to Sonar and store the artifact over at the Nexus repository.

If all that stuff is successful, which means we have a proper IPA, with the right provisional

profile sign, we ship that artifact out to Hockeyapp for QA to be able to download and

take a look at it. [58:41] Rudy >> So here is some of the individual

artifacts we have nest code built. That does the actual build. We have the code quality

metrics that we run right before we run the build. So I’m gonna open up one of the jobs

where you can actually see how it’s tied together. So this is an example of it first,

so first we do the code quality, if it’s successful we move on with Xcode build and

if that is successful, we move on to storing it in the artifact. We jot that point where

we contact Nexus and we ship out the artifact for storage and retrieval and we sign it with

the proper artifact and we store those. [59:19] Rudy >> The builds themselves take

about five to ten minutes, so with all this stuff included, so from the time that a…

go ahead sorry… [59:27] Audience >> When does the get figured?

[59:30] Rudy >> So we have two versions of it, right. We have a calling system which

we keep calling the repository every ten minutes. If there are new commits, we shake it all,

we build it. Automatically. [59:41] Audience >> On any branch?

[59:30] Rudy >> Yeah we have a few active branches that we have job set up for then

we have master and kind of stable branch which is always production-like, plus, the other

option is QA can always trigger it, if they want to. So, those are the options we have

built, and then we’re also working on refining what the proper incremental timeframe should

be for polling. Is it too frequent or not frequent enough, right? Because if the builds

take about ten minutes you don’t want to keep pulling every five minutes. Then you

have your jobs lined up and queued up and it’s going to bog down Jenkins itself. Any

other questions? [59:41] Audience >> [inaudible]

[59:30] Rudy >> So what we do is, with the code quality metrics… so after everything

is done, you’ll see that the build is successful and this is basically Sonar. So what we do

is we gather all the statistic from unit testing that job went through and we report it to

this tool called Sonarcubes. What it does is that it basically aggregates it and you

see the two green graphs for each branch right now? So, as we keep working on branches and

committing code, the code quality is gone, and reports are getting generated and updated.

If you start missing your commitment – let’s say threshold is set up, 90% for unit test

coverage and you add ten new classes and you have no unit test, you’re obviously gonna

cut off on that threshold and not meet it. So, we’re gonna flag it and once we ship

the results here, you’re gonna see red and yellow and going all the way down to red obviously.

We can actually enable triggers on these jobs, on this reporting, and say in this case we

have actual coverage set to, if it’s greater than 65%, mark it as warning. If it’s greater

than 70%, fail the build. So when the build fails it gets kicked back to the dev team

and they’re like, ‘alright we gotta address this, this is not making it to QA’, because

the next step is actually the build process and that never got triggers, so QA has nothing

to test on. [61:40] Luke >> Let me touch a little bit

on that. So, from the perspective of having metrics on how well you’re doing as an engineering

organization, this dashboard is very important to me, because we have multiple teams, we

have multiple, you know, we got Android, we got iOS, we got backend, we got multiple branches.

Setting rules and having these static code analyser rules that the team overtime develops,

is very important, right? Because those are essentially your automated code reviews that

happen every time you build, right? So as you’re developing these you’re getting

better and better and better, so you can make this a part of your retrospective process

if you want. You can make it more collaborative when developers are updating it all the time.

The bottom line is that those are very important. Another metrics I look at is comment density.

You wanna make sure that your code is well documented. I’m not big on writing separate

documents from the code. I think the code in the class is very important, also, that’s

on top of naming standards and conventions or whatever. Obviously, if you name your classes

correctly, if you name your method correctly, you don’t need comments, but if you don’t,

then it’s a problem. [63:06] Luke >> Duplications, that’s very

important for me as well. I don’t want copy and paste, right? It’s very easy if you

are bending out stuff. You’d be like, okay, that looks cool, I’m just going to reuse

it, copy and paste. If that goes up, I’m not a very happy person, right? And then,

that’s something that we are trying to make work, Cyclomatic complexity is also a very

good metrics to look at anything greater than 5, I usually reject the build, so that’s

analogous with having god objects, right? If I have a class with ten thousand lines

and does everything, cyclomatic complexity goes through the roof. I want nice, clean

eco-position, right? So, cyclomatic complexity is one way to ensure that you have that. And

then, our good old unit test coverage, so obviously, we want this number around 80%.

You’re going to have 100% coverage. You may strive for that but I use 80-20 rule.

80% of code coverage is good. The bottom line is that if you set the threshold and if you

setup your failure SLA, you’re kind of saying, ok, engineering, we have 60% now. From now

on you cannot decrease this number, you can only go up, which means, now you’re forcing

your developers to write code that has proper unit tests in place, and that’s part of

your automated process. You don’t have to rely on humans to decode reviews of whatever.

It’s part of your build pipeline. [64:48] Rudy >> You can kind of have, yeah

go ahead… [64:51] Audience >> Does Sonar sit on top

of your repository as if on github? [64:56] Rudy >> No, this is a separate thing,

this is a separate application. So what it does is, when Jenkins builds it, and it runs

your unit test coverage, it actually publishes the numbers to Sonar. Sonar takes them, figures

out what to do with them and does the whole graphing for you. So it’s kind of creating

the link between Sonar and Jenkins unit test reporting and it just takes care of everything

else. [65:17] Audience >> So when Jenkins builds

it, it sends it to Sonar? [65:21] Rudy >> Sonar, yeah, It spills the

reports to Sonar. Actually, I’m gonna do one thing for the demo right now and then

I’ll carry on and you guys can see the difference so I’m gonna go to alerts, and you noticed

how to coverage was. I wanted to fail, it gave me a warning at 80% and failed at 90%,

right? [65:52] Rudy >> So I’m just gonna trigger

a build right now and we can continue. [66:05] Audience >> So where is it building

from? Is everybody ready with a common file? Is there a server somewhere?

[66:11] Audience >> No, this is checking out the code repository, right, a given branch

and running the unit test coverage and doing the reports. If that is successful and Sonar

doesn’t report a failure on that, which means you haven’t exceeded any of the thresholds,

right, you’re still good, and then Jenkins carries on with doing the build, and then

storing the artifact. [66:29] Lukas >> We set up a pipeline, build

pipeline templates through Jenkins, so there’s a wizard that we have that allows you to specify

the branch. When you enter the branch, you enter the provisioning profile and that sets

all those builds for you with all those multi-steps building process and then you just schedule

and run. So let’s say I have new… I’ve been developing on my feature branches, I’ve

been checking through Stable. I’m ready to, okay, I’ve got enough stuff to do a

release candidate. I would cut the release branch, and then I would go to Jenkins, use

my wizard to set up with this branch build and now our building gets correct. There’s

also a stable branch that we’ll build against all the time. And that’s all, you can lock

it. [67:18]Rudy >> So the moving pieces where

getting a build ready for QA is basically checking the code, having proper unit test

coverage. We run the unit test coverage. If everything passes, we run the build, we create

the artifact. If that is successful, we store it over at Nexus and that’s the raw APP

file, right? So at that point we can sign it with any provision profile we need which

is an issue that Lukas mentioned that they were having earlier, which is once QA signs

off, the developer checks out the code again and does a brand new build and that’s all

gone now. We have the actually raw APP that QA signed off on depending on which provisioning

profile we use. If they sign off on that, we check out the exact APP file. We sign it

with production and we ship that same thing off to Apple.

[68:00] Rudy >> Now, dev could have messed around with the branch as much as they wanted.

That one build is going out, the one QA signed off on, right? There’s no more ifs and buts.

Did we check out the right stuff, did we build it properly. Did you guys update any libraries

locally before building, so… [68:17] Audience >> [inaudible]

[68:25] Lukas >> So there is a difference between unit testing, remember unit testing

is… automation, the functional testing automation, that’s a different topic, we’re going

to have a tech talk on that end of July. [68:40] Audience >> When you are running the

simulator for all your unit testing, do you have running wishes that it’s of the actual

device like integers? [68:51] Lukas >> So testing of the simulator

is not going to ever be the same like testing on the real device. So again, when you’re

setting out the functional automation framework, you want to make sure that you’re using

an actual device arm. There are companies that allow you to do that. Or you can have

the fondal devices in-house but you’re stuck managing that stuff, again that’s an end

of July session. [69:19] Audience >> That’s right, and what

about the iOS expression of iOS7 and they said, you can do iOS7?

[69:24] Lukas >> In the iOS7 you’re usually [inaudible]

[69:28] Audience >> Because that’s where everybody presents you so much.

[69:32] Lukas >> But we have a QA process we test against different devices.

[69:36] Audience >> Or can be just different versions of it?

[69:40] Rudy >> So that’s part of the build deployment process, basically, once all the

builds are ready to go and there are no issues, we… so this is Nexus. I don’t know how

many of you guys are familiar with Nexus. It’s basically a artifact dependency management

system that we put in place. It’s open source. You guys can utilize it as well. What we do

is basically after all the artifacts are ready to go… I’ll show you guys this version

actually. [70:13] Rudy >> We store them with the…

can you guys see this? Yeah, with the version number and the build number that came through

and if you guys actually expand that, you have multiple versions. You have the Ad Hoc

IPA, you have the Devord IPA and then you have the Tor Bold which is the actual raw

App file. So once QA signs off on this version, let’s say, which they did, you check out

the Tor Bold which contains the raw App file. We sign it with production. We store them

here so we know exactly what we released and we ship this one off to the dev, to Apple

Store. So if they have messed up with the branch at any time while QA was testing, 2016-2,

nobody is going to get impacted by it. We’ll have to do some branch cleap-up later but

the same artifact is gonna make it to production and into the app store. We also stored the

Git hash spirit, so at any point we can track it back and figure out this version was built

based on this Git hash, so we can check it out and figure out what went wrong, if anything,

right? [71:12] Rudy >> So QA will be doing… so

once that these are ready and they’re stored on Nexus, we ship them off to Hockeyapp, which

is the application distribution. Portal, basically. The devices connect to it, they download,

they install it, they go off testing. One of the issues we were having was basically

QA could never figure out which build they’re testing because they’re all 2016s, right?

We managed to do custom build numbers so they actually associate with individual Jenkins

builds so we can go back and say, oh ok, so version 5 is actually on Hockeyapp, it can

be downloaded and tested, so when they’re downloading, they actually see the latest

download of the version and they can correlate to know better, the build they’re downloading,

is actually from the last commit or is it from the previous one. Because the upload

to Hockeyapp could have failed, it might have, something might have gone wrong and they’re

just downloading and updating, they might never know. They get to download this stuff,

test it out. Once they’re good with it, we’ll go back to our Jenkins, we have a

process. [72:18] Rudy >> We have a process where we

basically prepare for production and what it does it basically give a version number

and a build number. It checks out the Tor Bold which concerns the App, signs it with

production profile and then uploads it back to Nexus which is available to download so

you can actually click on it. [72:37] Rudy >> Download it and submit it

to the Apple Store. Now this is done between the region’s management and the product

owners and we collaborate and figure out when the right time for release is and we just

go do this. So at any time, something that was tested sort of working through at this

point. Any questions, of any of this setup? [72:58] Audience >> Yeah sure, so you’ve

submitted for certification from Apple because Apple uses all of this still to verify [inaudible]

[73:16] Rudy >> Apple doesn’t care about Sonar reporting, yeah, Apple doesn’t care

about unit testing, any of that stuff. [73:21] Lukas >> Their evaluation criteria

is based on completely different things, right, versus what we want to do, so, again, tying

it back to what we talked about, you know, we want to enforce engineering standard code

quality. Architectural standards make sure that there is a separation between develop

and release management. There are tools to do that. We talked about Typhoon, Cocoapods,

all that stuff, but it all comes together, right, for you to be able to be comfortable

with experimenting and writing code and being in an environment where we fail fast. We learn

from that, we’ll fix it and then we’ll move on.

[74:06] Audience >> Those people use version dool very differently, in different places,

like [inaudible]. My question is this though, how do you actually, your backend, how do

you make sure that the new feature that you have, that you’re putting in there is always

connected to the backend. How do you tie it all together?

[74:29] Lukas >> So you’re talking about… So our application is powered by Respool APIs

which are versioned. It’s a completely different topic, how you manage your Respool APIs. All

I can tell you is that when it comes to like, testing, you know, we have a staging environment,

QA environment since we’re testing different versions of APIs. There are ways also for

you to set automated testing where you are essentially mocking their responses or mocking

their Jasons. You can use men in the middle or some other proxy where you are replaying

back some things, which kind of goes if you want to have atomic tests. That’s something

that we’re working on right now. It’s part of our automation plan.

[75:14] Audience >> So it’s not going to be automated before I tend to be line-correct.

[75:17] Lukas >> So we are doing a big push towards functional testing automation. Majority

of our tests are right now in the QA side manual and we use crowd-sourcing through YouTest

to test our application so as far as a road map is concerned, we’re a big push on the

engineering side to do tester involvement, increase test code coverage, and on the QA

side we also are just starting the automation effort so we’ll be applying some sort of

device farm. We’ll essentially be able to replace Kreps and simulate user behavior.

[76:01] Audience >> So it sounds like when you started this tester and development, you

had a premature app and there were no [inaudible] so how did you prioritize the coming unit

test build, did you put all of your engineering and development on hold while you, or 50%

or did you write for extra bugs as they came up?

[76:28] Lukas >> It’s really a Scrum discussion, right? How do you find that balance between

like building or engineering practice versus your product priorities and all of everything

else? All I can tell you is that we’re solving for stability and a better build of the app

and you know, when I’m having a discussion with my boss or with product, you know, it’s

kind of a horse trading thing where it’s like well, you know, we could be cranking

out a lot of features but if we’re crashing all the time and the app is not stable and

that’s not ideal. So you need to find that middle ground, right? The way we solved that

problem is that we said, ok, we’ve made unit testing part of our definition of done,

which means that if you’re building a new feature then you have to write, if you’re

attaching an old class, then you have to write a new test for that class. So over time your

test coverage increases, right? [77:34] Audience >> What is the engineering

responsibility? I mean, other than having me having say so, but what is the engineering

response to actually… [77:41] Lukas >> We actually made this decision

collaboratively. It wasn’t my mandate. I’m not a big fan of just being very heavy handed.

It was a discussion, right? We kind of weighed the pros and cons to different approaches.

At the end of the day, I think, all developers say, yeah it’s going be kind of sucky to

do this, but we had all gotten bored because at the end of the day we take pride in what

we do so our ship finally flowed. [78:13] Lukas >> Ok, no more questions. Yes,

sir? [78:18] Audience >> [inaudible]

[78:23] Rudy >> Yes, so it’s a separate topic. We are using Floury, we are using actually

several different libraries. Floury is just part of many. So, again, it’s kind of goes

with this motto of like, you don’t want to be in the business of writing all that

stuff from scratch, right? So if you want to get all this other stuff, you want to integrate

with some sort of a company that’s in the business of that so that leaves you free to

do other things, right? But that’s a separate topic.