Boring Optimizations

I think the most important change in 2.3 is that it uses a new, compressed storage format for the world files. I told you about it in one of the previous posts. It is a more complex system compared to the old one, which stored raw block values divided into constant-size chunks.

Here’s some details. The new format divides the world into regions of 16×16 chunks (where chunk is 16x16x256 blocks). Each region gets a separate file. The chunks within the region file are compressed using a two-step process:

Step 1. RLE compression to exploit the repetitive nature of the world blocks.

RLE compression stores the value of the block followed by the repeat count, instead of storing all blocks separately. The count value is stored in the “light” section of the block data, so it doesn’t take any more space than the raw, uncompressed block (light value does not have to be saved – it is generated anyway). Only if the count is too large to fit in the light section (4 bits), a separate byte is allocated for the count.

Additionally, the order of blocks storage is changed from how they are kept in memory. This is to maximize gains from the RLE scheme. Block values are more repeatable in horizontal directions than in the vertical. Deep down you get hundreds and hundreds of the same granite/basalt blocks, up in the air it’s all air, obviously.

Step 2. Standard deflate compression (the same as used by zip format) on the resulting RLE-compressed data

This does not need much explanation. Deflate has been the workhorse of compression for many decades now. There are more modern and better compression algorithms – like Zstandard, which is both faster and provides better compression. Maybe one day I will switch to using it. But Deflate is so ubiquitous and so easy to use – it is good enough for us at the moment.

The main complication with the new format is that chunks no longer have the same size – some compress more, some less. A simple fixed-size system, as used in 2.2, cannot work. Therefore, within each region file there is a mini-filesystem, which handles variable-size compressed chunks. It uses a simple overprovisioning scheme, which leaves extra 1024 bytes after each chunk to accommodate small changes. Once a chunk grows out of this extra space, the entire region file is rewritten, with the file enlarged and chunks rearranged to get their 1024 spare bytes again. The rewriting does not happen too frequently – a large change to the chunk data, such as an explosion or digging/placement of many blocks, is required to use up the 1024 bytes and trigger the rewriting.

The result is quite satisfactory. I’ve already shown you this:

2.78GB to 26MB – that’s over 100x reduction in size! But we badly needed that. The large number of crash reports I have from the game are almost always caused by the device running out of storage space. Followed by running out of memory, when someone with a low-memory phone ups the visibility range to the max…

Such changes are always risky – we don’t want world data losses. To reduce the chance of this happening, I used both automated and manual testing. For the automated testing, the world was randomly modified and camera randomly shifted in quick succession. This was left to run overnight – for many, many hours – stress testing the new format. For manual testing, me and friends played 2.3 in many different settings, with up to 4 players, for many hours in total.

As far as I can tell, it works. There are no bugs. But we’ll see once YOU start using it :-)

The new file format is not the only optimization in 2.3. Memory allocations in terrain engine have been massively reduced by use of caching. This should reduce garbage collection-related stuttering. In 2.2 and earlier, as the player was moving quickly, the chunks which were going out of visibility range behind the player were immediately freed, and memory for the new chunks was allocated afresh. In 2.3, the blocks of memory from old chunks are cached for a while to see whether the new chunks will fit in them, instead of allocating fresh memory.

Nothing too clever, but it hugely reduces pressure on the garbage collector. We had a system like that back in the old days of Alpha 1.xx around 2013, but I had to take it out, because back then memory was more precious than gold dust. The game had to run on some Apple devices with just 256MB of RAM. Fortunately, Apple has changed its ways, and the worst we can expect from them is 1GB on iPhone 6, or 512MB on iPad 2 – would you believe that scores of people still play on that ancient museum piece? That’s the sole reason I keep supporting iOS 9.x.

There’s more optimizations to do with light propagation as well. As usual, the full list of changes will be published soon on the Updates History page.

By the way, if you read all the way to here without getting bored: I released 2.3 earlier today on all platforms. Should already be in the stores, apart from Amazon, which is still in review. Amazon will probably be out tomorrow.

Have fun!

77 Comments

  1. Hank Sauri
    Posted February 15, 2022 at 15:20 | Permalink | Reply

    Optimizing improves the experience for everyone though

    • Stanimus
      Posted February 16, 2022 at 01:36 | Permalink | Reply

      Not really. I had no problems at all the way it was. Now I’m hearing about all kinds of problems with lost saves, crashing devices and corrupted words after being saved. No. Definitely not an improved experience for many!

      • Posted February 16, 2022 at 03:31 | Permalink | Reply

        Hi @Stanimis. Been hoping to run into you here. Hope you’re doing well. I’m glad I’m out of the manual SC hacking game as this news invalidates all the code I’ve ever written!

        • Stanimus
          Posted February 17, 2022 at 23:17 | Permalink

          Hey Neo! It’s great hearing from you. I’ve had some rough times recently but still hanging in there. And still playing with this game! I’m quite active on the discord. There’s been a couple peeps there doing some pretty heavy ‘hacking’. One guy has done some impressive things just editing the project file but that stuff all disappears when the world is saved. I’ve got enough done in my CPU that I can honestly say “It works!” However I still don’t spend enough time on it to get it to a presentable state.
          Hope life has been treating you well enough. I know your work can be stressful…

  2. Posted February 15, 2022 at 15:25 | Permalink | Reply

    Over the past couple of years. The Survivalcraft Content Community has continued to grow!

    Over the past few days we finally reached 1000 discord users. We’re now at 1046!
    The server has been a welcoming place for the English speaking community to still discuss and celebrate the game in times of developmental inactivity. With users showcasing their builds, videos and other projects with other fans of the game.

    We’ve been talking about how 2.3 was appearing in app stores awhile before this post was made. We also get automatic updates from Kaalus’ Youtube and Twitter account. It’s a great way to stay up to date!

    You can join the discord by typing discord(.)gg /Survivalcraft into your browser’s address bar. Or by typing “Survivalcraft” after clicking the join server button in discord.

  3. Posted February 15, 2022 at 15:26 | Permalink | Reply

    There is a bug in the menu, after the update it says “Welcome to survival craft 2.2” instead of 2.3

    • Hank Sauri
      Posted February 15, 2022 at 15:27 | Permalink | Reply

      Noticed the same issue

  4. Baraa Alhorani
    Posted February 15, 2022 at 15:28 | Permalink | Reply

    That wasn’t boring tho

  5. Posted February 15, 2022 at 15:28 | Permalink | Reply

    I definitely agree that the new dramatically reduced world file sizes was the most important feature this update. But I also personally love the birds! The sparrows add nice ambiance to the world. Good job and I’m glad to see you back!

  6. Posted February 15, 2022 at 15:32 | Permalink | Reply

    There is a working multiplayer version of the game built by a small group of Chinese modders that are eager to get in touch with you.
    It’s not perfect, but it works, and it works relatively well at that.

    They haven’t released it publicly. And seem to want to do things right by contacting you. I’ll leave a forum link replied to myself below. I’ll also send more details over email tomorrow.

  7. Null Void
    Posted February 15, 2022 at 15:40 | Permalink | Reply

    Issue with pickables, Special K!
    Big explosions create far too many pickables for even new phones to keep up. old ones would crash for sure. My fps dropped to 1 for a solid 30 seconds.

    I suggest you add pickable stacking. So adjacent pickables will merge to prevent overwhelming the device. Too many particles!

  8. Milton Rivas
    Posted February 15, 2022 at 15:45 | Permalink | Reply

    is there a way i can get the game by paying with paypal? I have no credit card and microsoft store dont allow me to use paypal neither google because my country is not supported they said

  9. Null Void
    Posted February 15, 2022 at 15:55 | Permalink | Reply

    Small suggestion… Pigeon flocking behavior. Birds dont behave like a herd… Or flock… They just seem unrealistically independent of each other.

    • Null Void
      Posted February 15, 2022 at 15:57 | Permalink | Reply

      Birds dont eat either… I want to feed the new birds some bread… And the ravens some rotten meat. Seagulls might enjoy a good sea urchin? And how about a fresh fish for those ducks?

  10. Posted February 15, 2022 at 16:00 | Permalink | Reply

    ive found a bug. check your email.

  11. Johan Steyn
    Posted February 15, 2022 at 16:11 | Permalink | Reply

    Sad, again no mp…

    Great work done though.

  12. Johan Steyn
    Posted February 15, 2022 at 16:22 | Permalink | Reply

    When mp Kaalus? Please give us an idea?

  13. Nathan Hohnberger
    Posted February 15, 2022 at 16:23 | Permalink | Reply

    Are you going to disappear for a year??? Now that this has been released? Man we are going to miss you!!!

  14. Posted February 15, 2022 at 16:29 | Permalink | Reply

    Hey!

    I’m glad we finally can have a new update after all that time, but I think that you might lose what’s left of the game’s community if you continue only giving news and small updates every two years…
    I loved Survivalcraft and still do, but I almost don’t feel the need to play amymore as there isnt much exciting stuff left to do in the game.

    I mean we all waited 2 years without any update or even a single news to know you’re still working on it or not, and all we get beside the world optimisation is two birds (which are useless beside adding nicer ambiant sounds), 3d items, a new bomb and other small details?

    Thank you for taking this into consideration, I will definitely play the new update to tru the changes, but will surely get bored out after a couple hour knowing I might have to wait another two years before anything new…

    • Johan Steyn
      Posted February 15, 2022 at 16:36 | Permalink | Reply

      I have updated, tested my world and 3 minutes later quit again. Without MP, SC is unfortunately dead. I do however enjoy the mods that is available. If Kaalus is not going to give MP, I’ll be playing the Chinese version. Sorry for sounding negative. I do think Kaalus is a great programmer and the game he developed is wonderful. I have defended him many times against criticism, but I think I am done until MP comes. You can only play so long with your own toes…

      • Johan Steyn
        Posted February 16, 2022 at 11:00 | Permalink | Reply

        Sorry I was a bit negative. I suppose Kaalus is free to do with his game he wants to. If he never wants to bring MP to SC, it is his choice.

  15. Posted February 15, 2022 at 16:31 | Permalink | Reply

    Good morning! Just read it all, thanks for sharing and now I’m off to download!

  16. Stefix1
    Posted February 15, 2022 at 16:36 | Permalink | Reply

    thank you.

  17. Posted February 15, 2022 at 16:40 | Permalink | Reply

    Intro slider still says “Wlecome to 2.2”, version is 2.3.60.0, Apple.

  18. Posted February 15, 2022 at 16:48 | Permalink | Reply

    launch the game on steam, please, i am beggin you

    • Posted February 16, 2022 at 16:59 | Permalink | Reply

      Yes, I also really want Survivalcraft to be released on steam.

  19. Posted February 15, 2022 at 16:49 | Permalink | Reply

    Please before you disappear for years again

  20. Muhamad Auni
    Posted February 15, 2022 at 16:56 | Permalink | Reply

    This change is basically about optimization.

  21. Null Void
    Posted February 15, 2022 at 17:13 | Permalink | Reply

    I have a small suggestion. A slider in world settings for
    an explosion power factor. Up to 2x? Down to 0.5x?

    This would make for an incredible piece of customization. No need to change the values anymore. We could do that ourselves.

  22. travis dent
    Posted February 15, 2022 at 17:47 | Permalink | Reply

    Will the new cairns also appear in old saved worlds when upgrading them to 2.3? I see a huge file size improvement which I expect will save people running out of space due to 2.2.

    Is there any benefit to digging up cairns? Seems like only experience orbs are obtained?

  23. 松雪まふゆ
    Posted February 15, 2022 at 17:55 | Permalink | Reply

    When I am in a 1×1 space, I will die if I am attacked by another animal, including creative mode.Could it be a bug?

  24. Posted February 15, 2022 at 17:55 | Permalink | Reply

    Noticed we can’t upload a world link from dropbox to community content anymore, displays an error message when trying to create and copy a link from the dropbox file… Anyone having the same problem?

  25. 金蛇Goldensnake
    Posted February 15, 2022 at 18:06 | Permalink | Reply

    Kaalus, there is adeadly problem!If the player gets into a block high hole and is attacked or coughs and vomits, he will die of chock. In addition, the third person squat was not done. Also, if the pipe entrance is in front of my upper body, I will not be able to enter by squatting and jumping.

  26. 金蛇Goldensnake
    Posted February 15, 2022 at 18:11 | Permalink | Reply

    It is a great departure from common sense that we get rotten meat from sparrows, pigeons and piranhas. Mr. Kaalus, I hope that even if they do not produce any resources, they should not drop rotten meat. Besides, pigeon is a kind of high-quality food material all over the world and should not produce nothing. In this case, it may be more suitable to put it on ravens or seagulls. Even sparrows taste better than ravens and seagulls.
    If possible, this would be a better choice: sparrows and pigeons have a 50% chance of producing raw birds, while seagulls and ravens produce carrion.

  27. 金蛇Goldensnake
    Posted February 15, 2022 at 18:25 | Permalink | Reply

    Besides, in any case, I didn’t find diamonds from carins as mentioned in the updated list.

    • 金蛇Goldensnake
      Posted February 15, 2022 at 18:26 | Permalink | Reply

      ahh yeah cairns

    • Stanimus
      Posted February 15, 2022 at 20:21 | Permalink | Reply

      So far, every report I’ve heard nobody has found a single diamond in a cairn. They apparently mostly have sticks, rocks and maybe some exp.

  28. l0st5ilver
    Posted February 15, 2022 at 18:38 | Permalink | Reply

    BUG: I was disappointed to see that, in light of the new crouching features, the old glitch where you die riding a horse under a tree has not been fixed. I would greatly suggest a hotfix to remove this issue from the game once and for all, as it has ruined many a player’s experience.

  29. Posted February 15, 2022 at 18:57 | Permalink | Reply

    Wellcome back Kaalus my dude! Glad to see that you are okay, and glad to see that SurvivalCraft isn’t dead!
    Thanks for you efforts 😊

  30. Posted February 15, 2022 at 19:56 | Permalink | Reply

    Have yet to spot a single cow. Is that typical for 2.2/2.3?

    • travis dent
      Posted February 15, 2022 at 20:32 | Permalink | Reply

      Do you have your ‘COWS’ slider set to NONE?
      If cows are not spawning, you may need to increase that or go all the way to MANY HERDS (the opposite end).

      Also good time to check ‘MILK OUTPUT’ slider and make sure it’s not set to INFREQUENT.

  31. Revision
    Posted February 15, 2022 at 20:16 | Permalink | Reply

    I found a bug, it looks like you can see caves and lava pits through a mountain.

  32. Stanimus
    Posted February 15, 2022 at 20:18 | Permalink | Reply

    By making the motion detector trigger on pistons, this has just broken every one of my Doom worlds!

    • Stanimus
      Posted February 15, 2022 at 20:32 | Permalink | Reply

      This really screws me in making adventure maps. I’ve relied on using the piston to BLOCK the motion sensor to PREVENT unwanted triggering. Now the will cause the very thing they were meant to prevent!
      Ugh

      • Posted February 15, 2022 at 21:39 | Permalink | Reply

        Use a trapdoor or a gate to prevent unwanted triggering. Easy. This feature has more upsides than downsides

        • Stanimus
          Posted February 16, 2022 at 01:58 | Permalink

          It is sooo not that easy. You clearly don’t understand the problem.

        • Stanimus
          Posted February 16, 2022 at 17:01 | Permalink

          Here’s just one example:
          A common application is to use the detector so it triggers after someone has walked through a piston door. Then it shuts that door. Without changes, the door just opening up will trigger the detector, which will shut the door (and may even lock it) thus totally ruining the plot.
          In many cases (but not all) i can work around this by narrowing the view from the detector.
          And this is only one instance of where it’s a problem.

  33. A Nice guy
    Posted February 15, 2022 at 21:04 | Permalink | Reply

    my worlds were gone after this update all the building blocks on my map HAVE BEEN DELETED WHAT ANGER

    • Stanimus
      Posted February 16, 2022 at 16:57 | Permalink | Reply

      Did you neglect to save all your working worlds before you updated?

  34. t5670012
    Posted February 15, 2022 at 21:43 | Permalink | Reply

    I think pigeons and sparrows should drop small bird meat with a small chance and crouching should have its own animation. It’s good that you continue the development of Survivalcraft 2!

  35. Null Void
    Posted February 15, 2022 at 22:44 | Permalink | Reply

    The new cairns dont drop diamond… No matter how many I place… I have been tapping for 15 minutes, and no diamonds. Im pretty sure they should drop a few. Just sticks and rocks…

  36. Null Void
    Posted February 15, 2022 at 22:44 | Permalink | Reply

    The new cairns dont drop diamond… No matter how many I place… I have been tapping for 15 minutes, and no diamonds. Im pretty sure they should drop a few. Just sticks and rocks…

  37. Yucando Villanueva
    Posted February 16, 2022 at 00:12 | Permalink | Reply

    Error found!, when you crouch in a space of 1 block, and then pause the game, when removing the pause… you die from asphyxiation, greetings

  38. boostguardian1127
    Posted February 16, 2022 at 01:23 | Permalink | Reply

    Man, i was really looking forward to this update, but after reading the update history and actually playing it myself, i am severely disappointed. We have waited 2 years of silence and the only new things of note are the birds, smaller world sizes, better blasting potential, and cairns on mountains. Not only is this a small amount of things, but half of them arent even worth it. The new birds are nice to see and are cute, but are rather useless. No food drops, which was intentional for some reason, and their eggs basically give you nothing. Secondly the cairns. I appreciate giving mountain trekking a purpose, but the rewards are not worth it. They give no diamonds at all, and only give enough experience to fill one bar in the experience meter. Its not worth freezing to death for such little pay off. I love survivalcraft so much, but lately its starting to become more of a chore to play. I think im gonna be sticking to survivalcraft 1 for the time being. I hope Kaalus sees this and takes it into consideration.

  39. wp1729913896
    Posted February 16, 2022 at 03:13 | Permalink | Reply

    Will it be available on the Steam store?

  40. Revision
    Posted February 16, 2022 at 04:02 | Permalink | Reply

    Kaalus: Ahh, lan multiplayer is for the birds!

  41. Revision
    Posted February 16, 2022 at 04:02 | Permalink | Reply

    🤣🤣🤣

  42. wp1729913896
    Posted February 16, 2022 at 08:45 | Permalink | Reply

    Due to a special policy, we need to get the Android version through Steam.

  43. Baraa Alhorani
    Posted February 16, 2022 at 09:45 | Permalink | Reply

    Guys i have a theory why the update wasn’t as many expected, when kaalus posted after two years of disappearing he wasn’t working on survival craft all that time, but he mentioned that he was busy with other things and now he back to get us some new exciting things ( i hope) so guys please sending disappointment messages for him and instead try to tell him with someway to exceed the disappointment, and i know everyone playing survival craft for so long disappointment and so do i, but kaalus wasn’t really working on survival craft for two years, and thanks, i hope next updates everyone will like it and enjoy it.

    • Baraa Alhorani
      Posted February 16, 2022 at 09:47 | Permalink | Reply

      Stop sending**

  44. Ricardo Pereira
    Posted February 16, 2022 at 15:20 | Permalink | Reply

    Sorry for posting in the wrong place but I don’t know where to post bugs. Since I upgraded my laptop to windows 11 I can no longer ride horses and board boats. If Mr Kaalus sees this and can help me I would be grateful.

  45. Stefix1
    Posted February 16, 2022 at 19:42 | Permalink | Reply

    For those saying that they want MP, Kaalus did say before it could take X months and could potentially break the game. It would also make updating harder since you’d had to update the servers. The worst part is the servers though. They could be hard to set up and the game was never made to be compatible with servers. So just stop begging, let him work on the game himself. It was planned since the first version (in the first video he made about survivalcraft) but it turned out to not work. You’ll have to wait.

    • Baraa Alhorani
      Posted February 16, 2022 at 21:25 | Permalink | Reply

      LAN WiFi doesn’t need servers to adjust tho

    • Posted February 17, 2022 at 03:24 | Permalink | Reply

      At this point I doubt that anyone here want mp with servers running by kaalus everyone want lan mp and if we want to play online we can mount virtual lans

    • Johan Steyn
      Posted February 17, 2022 at 15:50 | Permalink | Reply

      Sorry, but has been almost a decade. And the Chinese group has already written a mp mod for SC that is actually working and they even contacted him about it. He has not responded as far as I know. Hopefully he will.

  46. Fernando D Oliveira
    Posted February 16, 2022 at 20:07 | Permalink | Reply

    I guess one must be careful to update to 2.3 before loading a 2.3 map. At Windows 10 Store the version did not upgraded automatically at some systems. So a map already upgraded to 2.3 was loaded into a 2.2 version and worked, BUT LATER when loaded again at another Windows 10, it crashes.

  47. rafael304
    Posted February 16, 2022 at 20:32 | Permalink | Reply

    Posting here to make sure for you to see my email! Its a very critical problem, that region file system uses a folder but it doesn’t is included in world files,

    So, If who reading this is not kaalus, Don’t upload worlds until that is fixed!
    That is, Don’t upload/import worlds to/from dropbox/transfer.sh until that gets fixed.

    There is others problems, So, Please, See your emails candyrufusgames @ gmail(.)com!

  48. Олег Губайдулин
    Posted February 17, 2022 at 09:19 | Permalink | Reply

    If you dig a long narrow (1×1) tunnel, you have to climb out every time. Otherwise, you will die when you re-enter the world. Since the spawn is in full growth.

    • Олег Губайдулин
      Posted February 17, 2022 at 10:06 | Permalink | Reply

      And yet, in 2.3 we cannot use “pause” when something distracts us, during a pause you can be killed or you will freeze for example.

      • Олег Губайдулин
        Posted February 17, 2022 at 10:10 | Permalink | Reply

        The new sensitivity of the detector does not always work. Standing in one place and in one direction, I fired arrows, snowballs, threw objects … it works from 70% to 90%.

        • Олег Губайдулин
          Posted February 17, 2022 at 10:22 | Permalink

          may not even notice the werewolf. Of course, there is still a lot to test. I am very pleased with the stability of the new save and synchronization system. The game has become … difficult to find words … well, it’s faster and easier. Thanks to Kaalus for the work, but I think there will still be shortcomings. We need more tests. This is what the community is doing. But we need more time.

  49. Posted February 17, 2022 at 10:05 | Permalink | Reply

    Nessa nova versão, quendo eu coleto itens ele trava

  50. Wafity
    Posted February 23, 2022 at 17:12 | Permalink | Reply

    Hey I have been playing survivalcraft for about 3 years, since 2016 mid 2017, I started to play it more often now since I have noticed that compared to minecraft, the survival mode isn’t real survival, since you can make all the dangers go away and the game gets very boring. On the other hand in Survivalcraft there is always a sense of danger nearby, torches wont protect you from wolves, lions or tigers and I love the challenge! I have to always be on the lookout for wolves and lions and get into my house quickly at night, or else I’m bait! Minecraft gets very repetitive once you kill the Ender Dragon and have full diamond armor, the whole survival aspect is gone and its boring, like yeah theres achivements but its such a grind. I love Survivalcraft way more because the game is challenging and never gets too easy, like you can never be 100% safe, even in your own base, especially at night. I could go on and on with comparisons between the two games but I think the name “Survivalcraft” really fits well, since the focus of the game is on survival, hunting, food and not just some health and hunger bars that go down inconveniently. I will say that mining for reasources in your game feels much more rewarding than in minecraft because of just how versitile copper is. I also love the different gamemodes that certain players may prefer based on their play-style and experience. Like I always used to play in Harmless mode but I wanted more of a challenge and so I tried out Challenging mode and I now always have to be on the lookout for predators, especially at night. Like the day time in Survivalcraft is full of dangers if your not careful unlike minecraft where zombies burn and remove the entire danger aspect of survival mode. All in all, the game is well worth the 4 dollars as it has so much more to offer than a game owned by Microsoft. It still blows my mind that this game manages to have more bugs fixed in a single or more updates than a game owned by a multibillionaire company. I think in conclusion that your game proves that with a huge amount of passion great things can be made, such as Survivalcraft. Keep working on this wonderful game Kaalus!

Leave a Reply. No swearing please.

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s