More 2.3 bugfixes

I have finished work on the new bugfix release in the 2.3 series. It fixes the following issues:

• Framerate drops due to sound effects playback on Android devices
• “Adventure Restart” functionality not working properly with adventure worlds created in older versions of the game
• External keyboard & mouse control not working on mobile devices

The slow audio playback causing framerate drops on Android has always been there, since 1.17 release back in 2012, the first version available on Android. But, as framerate in these ancient times was jittery anyway, the drops weren’t very noticeable. It gradually started to become a problem recently, as Android devices matured, and the framerate was more likely to stay at 60fps+ all the time.

I made a tentative fix to Survivalcraft 2 full version a few days ago, and submitted it to the store as a test. Unfortunately, the sound issue did not go away, the lags are still there.

So I decided to go nuclear. I completely ditched the current audio system for Android, which used Java-based Audiotrack objects, and replaced it with OpenAL. OpenAL for Android uses native OpenSL ES backend, instead of the problematic Audiotrack. Survivalcraft also uses OpenAL on all iOS devices, so the change wasn’t too complicated or too risky.

I submitted the new build of Survivalcraft 2 Day One (version 2.3.11.3) which contains the latest fix. Please update and let me know if everything works. If it does, I will submit the full version of Survivalcraft and the remaining games as well.

Please note – currently only Survivalcraft 2 Day One has the fix. The other versions of the game still use the old audio system.

Networking issues

As you probably noticed, Windows Store version of the game stopped working properly with Community Content, Dropbox and Transfer.sh. The reason is very simple: TLS 1.2 is now required to connect to these websites.

Briefly, for those who don’t know, TLS (Transport Layer Security) is a network protocol, which secures your internet dealings by encrypting the network traffic. Versions of TLS prior to 1.2 are considered unsafe, so recently a lot of websites started requiring TLS 1.2, and will refuse connecting with older versions.

The Windows Store builds of the game use an old .NET runtime, abandoned by Microsoft circa 2015, which has a TLS 1.0-only implementation of the network stack. Hence, Dropbox and others will refuse to connect to the game. This manifests with the following message:

The solution is to update the Windows Store version of the game to use the newer TLS. It means rewriting a lot of tested and working network code. Unfortunate and tedious, but necessary. Good news is, I have finished doing just that. The fixed versions of the game should now be in Microsoft Store:

• Survivalcraft version 1.29.47.0
• Survivalcraft 2 version 2.3.42.0

Please go to the store and update to get the fix.

2.3 Bugfixes contd.

I will very soon release a bugfix update for the 2.3 release. Here’s the list of changes:

• Fixed broken uploading/downloading of 2.3 worlds
• Crouching deaths in low spaces fixed, general crouching improvements
• Reduced Sparrows spawn rate
• Different, more varied new world names
• Fixed glitches and improved wooden ladder
• Fixed depth clipping when crouching near a wall
• Hammer makes appropriate material sound when used to make furniture
• Reduced chance of spawning of many creatures of the same type
• Reduced number of pickables created by explosions
• Fixed cairn collision box

The version numbers of the update will be 2.3.10.1 (Google Play), 2.3.61.0 (Apple), 2.3.50.1 (Amazon) and 2.3.41.0 (Windows Store).

We will need to wait for the stores to approve the update – hopefully only about a day. It used to be over a week back in 2013…

Some of you have been complaining that motion sensors detecting movement of projectiles, pickables and moving blocks (a new feature in 2.3) broke their builds, because they were using pistons to block off visibility of sensors. As pistons are moving blocks, they trigger motion sensors in 2.3 – so the blocking no longer works.

Motion sensor

I think this is unfortunate, but the fact the sensor detects more movement is good. What I would do, is to move away from mechanical blocking of sensors, and use the electronic way instead. Like that:

2.3 bugfixes

I am working on the usual bugfix release for the new 2.3 series. Thank you for reporting all the problems you encountered. The most important one is the new crouch mechanics having various problems, such as dying when attacked or respawning in the low-ceiling area. This is what I am working on now – should be releasing that and other fixes soon.

Boring Optimizations

I think the most important change in 2.3 is that it uses a new, compressed storage format for the world files. I told you about it in one of the previous posts. It is a more complex system compared to the old one, which stored raw block values divided into constant-size chunks.

Here’s some details. The new format divides the world into regions of 16×16 chunks (where chunk is 16x16x256 blocks). Each region gets a separate file. The chunks within the region file are compressed using a two-step process:

Step 1. RLE compression to exploit the repetitive nature of the world blocks.

RLE compression stores the value of the block followed by the repeat count, instead of storing all blocks separately. The count value is stored in the “light” section of the block data, so it doesn’t take any more space than the raw, uncompressed block (light value does not have to be saved – it is generated anyway). Only if the count is too large to fit in the light section (4 bits), a separate byte is allocated for the count.

Additionally, the order of blocks storage is changed from how they are kept in memory. This is to maximize gains from the RLE scheme. Block values are more repeatable in horizontal directions than in the vertical. Deep down you get hundreds and hundreds of the same granite/basalt blocks, up in the air it’s all air, obviously.

Step 2. Standard deflate compression (the same as used by zip format) on the resulting RLE-compressed data

This does not need much explanation. Deflate has been the workhorse of compression for many decades now. There are more modern and better compression algorithms – like Zstandard, which is both faster and provides better compression. Maybe one day I will switch to using it. But Deflate is so ubiquitous and so easy to use – it is good enough for us at the moment.

The main complication with the new format is that chunks no longer have the same size – some compress more, some less. A simple fixed-size system, as used in 2.2, cannot work. Therefore, within each region file there is a mini-filesystem, which handles variable-size compressed chunks. It uses a simple overprovisioning scheme, which leaves extra 1024 bytes after each chunk to accommodate small changes. Once a chunk grows out of this extra space, the entire region file is rewritten, with the file enlarged and chunks rearranged to get their 1024 spare bytes again. The rewriting does not happen too frequently – a large change to the chunk data, such as an explosion or digging/placement of many blocks, is required to use up the 1024 bytes and trigger the rewriting.

The result is quite satisfactory. I’ve already shown you this:

World in 2.3 format

World in 2.2 format

2.78GB to 26MB – that’s over 100x reduction in size! But we badly needed that. The large number of crash reports I have from the game are almost always caused by the device running out of storage space. Followed by running out of memory, when someone with a low-memory phone ups the visibility range to the max…

Such changes are always risky – we don’t want world data losses. To reduce the chance of this happening, I used both automated and manual testing. For the automated testing, the world was randomly modified and camera randomly shifted in quick succession. This was left to run overnight – for many, many hours – stress testing the new format. For manual testing, me and friends played 2.3 in many different settings, with up to 4 players, for many hours in total.

As far as I can tell, it works. There are no bugs. But we’ll see once YOU start using it :-)

The new file format is not the only optimization in 2.3. Memory allocations in terrain engine have been massively reduced by use of caching. This should reduce garbage collection-related stuttering. In 2.2 and earlier, as the player was moving quickly, the chunks which were going out of visibility range behind the player were immediately freed, and memory for the new chunks was allocated afresh. In 2.3, the blocks of memory from old chunks are cached for a while to see whether the new chunks will fit in them, instead of allocating fresh memory.

Nothing too clever, but it hugely reduces pressure on the garbage collector. We had a system like that back in the old days of Alpha 1.xx around 2013, but I had to take it out, because back then memory was more precious than gold dust. The game had to run on some Apple devices with just 256MB of RAM. Fortunately, Apple has changed its ways, and the worst we can expect from them is 1GB on iPhone 6, or 512MB on iPad 2 – would you believe that scores of people still play on that ancient museum piece? That’s the sole reason I keep supporting iOS 9.x.

There’s more optimizations to do with light propagation as well. As usual, the full list of changes will be published soon on the Updates History page.

By the way, if you read all the way to here without getting bored: I released 2.3 earlier today on all platforms. Should already be in the stores, apart from Amazon, which is still in review. Amazon will probably be out tomorrow.

Have fun!

Mountaineering

Here’s a new video for you, shot high in the coastal mountains!

More Birds

You’ve noticed it right away. Flat blocks in 2.3 are rendered in extruded 3D style instead of flat. This only happens while the block is held in hand – pickables and projectiles in the world still display as billboarded particles. It looks better that way, I experimented with it. The code to extrude textures into 3D models was written by my son Mikołaj.

These are not 3D models. They are automatically generated from the blocks texture.
If you load a custom texture, the models will match.

Some more new stuff: 2.3 will include two new bird species. A sparrow and a pigeon. I have long thought that it is too easy to feed yourself hunting birds or eating their eggs. To make it more difficult, these two new bird species are not edible, and they are quite common, competing with the edible species. They still lay eggs, but their eggs are small (pigeon) and tiny (sparrow), and do not provide as much nutritional value as duck, raven or seagull eggs.

Sparrow. A tiny, agile, cute bird which lays equally tiny but edible eggs. Makes little chirping sounds that must be familiar to everyone on the planet (unless you live in the Antarctic?). Not edible. Lives everywhere apart from the very coldest places.

Pigeon. A small bird with a very familiar song. Lays larger eggs than sparrow, but is also inedible.

As a result, survival is harder. But you get the new default Survival mode instead. I think I told you about it already in one of the previous posts?

Sparrow is so tiny it can easily be killed with a rock, you don’t even need a spear. But why would you do that, it’s so cute!

All bird models have been updated to make them slightly more detailed and smoother to better match other animals:

I am almost ready to release, just a few more things.

Can you spot it?

Can you figure out what’s new in this screenshot from 2.3?

2.3 is coming out soon

The release is imminent. I have been busy working on other projects in the past months, but there’s a lot of changes in 2.3 ready to be released.

I have long wanted to make mining with explosives more viable than it used to be in 2.2 and before. The problem with explosives was that they were too expensive to make. It took a lot of gunpowder, which in turn depended on rare sulphur, and even more rare saltpeter. More importantly, explosions had a large chance of destroying the minerals instead of letting you pick them up after the blast. This is changing in 2.3.

And electric detonator has a new looks too:

The restriction of using certain tools to mine certain minerals has been removed. Now, explosion is just as good as a diamond pickaxe. Minerals pickables are not destroyed by explosions any more – they might fly all around the place, but they will not disappear. Plus, natural rocks have been made less resilient to explosives. And powder kegs are more powerful. All in all, I think blasting will be the best way to mine quickly.

Here’s a little video for you showing the process:

https://youtu.be/1Uz_Pz_suJA

If you watch carefully you might notice some other new things coming in 2.3…

SC Editor

Hi Everyone. One of the members of the community has been busy working on a world editor for Survivalcraft. It’s called SCEditor and there’s a beta version available for download here: https://drsmcraft.github.io/SCEditor/

I haven’t had much time to test it myself, but it seems to be functional and looks like it has full editing capabilities, down to the smallest details like block data values or obscure world options. Kudos to the creator. It even allows you to load worlds directly from Survivalcraft data folder. It’s still a beta though, so things may not always work as expected. Give it a try!

I have been temporarily sidetracked from working on the 2.3 update by a new project that came along this spring. This project is another multiplayer strategy game, using similar network protocol as Ruthless Conquest does, but more complex and advanced in every way. I want to get it finished before I get back to Survivalcraft. That hopefully shouldn’t take too long. So far I spent about a month doing base tech and another month on the game itself. It seems that the Coronavirus lockdown is very conductive to hard work :-)