AI opinion on population trend

[abezverkhiy]

Sorry king but you're wrong on this one.

I admire how you play your way uncaring of what others think, but based on the industry practice of analysing a liveops game service, rorswar is correct here.

You have to account for seasonality.

Very simply put, imagine its the Christmas holidays. Number of players has increased. Probably because more people have time on their hands to play. Once the holidays are over, users plummet. Was it the game's fault or people just don't have as much time anymore?

To measure if the game did better, we compare what happened this Christmas and the Christmas before. This is an oversimplification but I hope you get the idea.

If the game did worse this Christmas compared to the one before, there does seem to be a game related problem.

Now the analyst brings out retention models to locate where the game is bleeding players. Generally in old live service games like RoR, the problem is usually found in deteriorating inflow of new users.In other words, New User Retention has decreased and is at a number which is not enough to maintain the same player base (Outflow > Inflow). Ok, new player problem identified, maybe it's time to improve FTUE and onboarding.

Now, how are the regular or recurring players doing? Lower than last period? Probably time to introduce new content/balance changes to shake things up.

My explanation is an oversimplification, there's much more to it and can be statistically significant only with all the data available. The current chart you have I believe is CCU which is indicative at best.

Anyway, even after all this work, the analysis is the easier part. Executing solutions is a completely different topic.

Mate, thank you for your comment. You and others get it wrong when you take it as an attempt of predicting a future trend or seasonal trend or whatever other trend. It is not about trends.

I already said it but perhaps it was overlooked so I will try to explain it using a real life example:
- Imagine you own a shop and you try to promote it. Promotion campaign #1 - TV ad, promotion campaign - radio ad, #3 - give away of some slow moving goods, #4 - smth else etc. And after some time you look at your sales chart, mark dates of every event and see their effect on your sales. Which ones increased your sales and which ones did not. Every business owner does that. This is all I was looking for. No seasonal trends, no outside events, no 5 years of statistically correct data. Nothing of that bull.

All I asked AI was - read the chart, plot patches on it, comment. If you haven't read - read it. AI said that population is reducing and we know it from the times of the patch that banned NerfedButtons but what AI also said - it picked up successful patches that did bring players back. This is the best part why I decided to make the post. Not because I am upset with WH nerfs as some forums troll says.

And next we have all these smarta$$es coming and telling what analysis should look like so that we all know they also went to school. Or still go to school. Good on them. They do not read because tldr, they do not think, they just flame. They are noise and should not be treated with much attention and regard.

[Feodorovna]

...who only comes here to raise your self esteem by pissing everyone over.

In my real life I look at numbers and trends for 20+ years so seeing a couple of silly comments told me that I forgot that not everyone has the same background and able to read and think numbers.

But that is OK, this is hardly surprising when seeing election results, news, social media etc. Same people we see there come to play RoR as well.

And next we have all these smarta$$es coming and telling what analysis should look like so that we all know they also went to school. Or still go to school. Good on them. They do not read because tldr, they do not think, they just flame. They are noise and should not be treated with much attention and regard.

The irony.

[rorswar]

I already said it but perhaps it was overlooked so I will try to explain it using a real life example:
- Imagine you own a shop and you try to promote it. Promotion campaign #1 - TV ad, promotion campaign - radio ad, #3 - give away of some slow moving goods, #4 - smth else etc. And after some time you look at your sales chart, mark dates of every event and see their effect on your sales. Which ones increased your sales and which ones did not. Every business owner does that. This is all I was looking for. No seasonal trends, no outside events, no 5 years of statistically correct data. Nothing of that bull.

This example is flawed as well.

When a shop introduces a marketing campaign, they introduce a small amount of variables, so you can A/B test variants of the campaign, and against previous campaigns, in order to isolate which of the variables drove changes in sales.

In this instance of RoR, you compare movements of population against big events and patches, where hundreds of variables were introduced - some with big impacts (eg AOE limit changes), others with smaller impact. Therefore, you have no idea what drove the population movements, other than that you are able to say: “Ooohhhh, population moved here, and it coincided more or less with a patch or some other sort event.”

You cannot introduce a degree of confidence which variable within that event actually contributed to it, so it’s just a generalised statement and no more.

Even then, you need to take into account seasonality as that needs to be isolated from the population movements to figure out the proper impact.

It’s the same with the shop. If they introduce a $1 discount - and no other variables - and sell more lemonade, they still need to take into account seasonal and external factors.

If not, they may conclude that the $1 drove the sales uplift, and consequently they may decide to discount in perpetuity, whilst the real contributing factor was that it was holidays, and more people are out and about. Or it was 35 degrees vs 25 last week, and thus people are more thirsty. Or it’s a special date, like 4th of July, and people are more in the mood for lemonade after a BBQ or whatever.

And even there, seasonality matter as did it sell more or less than a comparable period the years before, as it may turn out that there was a sales uplift, but relative performance was actually worse vs a comparable period?

That’s why data scientists are highly educated and highly paid people, rather than a forum warrior with ChatGPT.

[sharpblader]

Sorry king but you're wrong on this one.

I admire how you play your way uncaring of what others think, but based on the industry practice of analysing a liveops game service, rorswar is correct here.

You have to account for seasonality.

Very simply put, imagine its the Christmas holidays. Number of players has increased. Probably because more people have time on their hands to play. Once the holidays are over, users plummet. Was it the game's fault or people just don't have as much time anymore?

To measure if the game did better, we compare what happened this Christmas and the Christmas before. This is an oversimplification but I hope you get the idea.

If the game did worse this Christmas compared to the one before, there does seem to be a game related problem.

Now the analyst brings out retention models to locate where the game is bleeding players. Generally in old live service games like RoR, the problem is usually found in deteriorating inflow of new users.In other words, New User Retention has decreased and is at a number which is not enough to maintain the same player base (Outflow > Inflow). Ok, new player problem identified, maybe it's time to improve FTUE and onboarding.

Now, how are the regular or recurring players doing? Lower than last period? Probably time to introduce new content/balance changes to shake things up.

My explanation is an oversimplification, there's much more to it and can be statistically significant only with all the data available. The current chart you have I believe is CCU which is indicative at best.

Anyway, even after all this work, the analysis is the easier part. Executing solutions is a completely different topic.

Mate, thank you for your comment. You and others get it wrong when you take it as an attempt of predicting a future trend or seasonal trend or whatever other trend. It is not about trends.

I already said it but perhaps it was overlooked so I will try to explain it using a real life example:
- Imagine you own a shop and you try to promote it. Promotion campaign #1 - TV ad, promotion campaign - radio ad, #3 - give away of some slow moving goods, #4 - smth else etc. And after some time you look at your sales chart, mark dates of every event and see their effect on your sales. Which ones increased your sales and which ones did not. Every business owner does that. This is all I was looking for. No seasonal trends, no outside events, no 5 years of statistically correct data. Nothing of that bull.

All I asked AI was - read the chart, plot patches on it, comment. If you haven't read - read it. AI said that population is reducing and we know it from the times of the patch that banned NerfedButtons but what AI also said - it picked up successful patches that did bring players back. This is the best part why I decided to make the post. Not because I am upset with WH nerfs as some forums troll says.

And next we have all these smarta$$es coming and telling what analysis should look like so that we all know they also went to school. Or still go to school. Good on them. They do not read because tldr, they do not think, they just flame. They are noise and should not be treated with much attention and regard.

No king, its not about prediction. It's about measuring the performance of anything introduced in the game and taking future decisions.

The problem is attribution.

Your e.g. is somewhat ok when it comes to measuring "sale of a product". Its easier to attribute it to specific actions if you can successfully link a buying user to a specific campaign.

But...I'll ask you this...if you do not know if people buying your product even saw the ad, or listened to it on the radio, how can we say with confidence we should do it again? Perhaps by chance there was a single high paying customer that found your product at the same time and referred their friends as well? Has it ever happened to you in the past where a campaign did well once but not the second time? Or did better the second time?

If attribution is proven in a simple user flow, e.g. Saw specific campaign -> clicked on the offer -> Made a purchase and once aggregated the specific campaign had the highest conversion and contribution to your revenue, by all means we should stick to it.

But when it comes to an in-game event, the player journey is much more complex. Its not as simple as buy or not buy....How many people actually participated in the event? How would you even define participation? Just collect the first milestone reward or the first picnic basket you got as a drop? Are your friends (or rivals) online to play it with you? Are the rewards BiS (for e.g. the annulus ring)? Can a player earn a lot of gold through it? And so on...

On the patches that did bring players back, there are 2 things we have to ask and confirm:
1. Was it the new content or we naturally get a surge of players in May every year?
2. If it is the content that is bringing back players can we realistically launch similar content each patch? Will dev resources, in-game economy balance and power creep become a blocker?