Jump to content

[TOPIC: topicViewTemplate]
[GLOBAL: userSmallPhoto]
Photo

Hosting the source
Started by johnfl46 Jan 09 2017 04:51 PM

16 replies to this topic

Best Answer dyson122 , 23 January 2017 - 06:35 PM

I developed MTE as a single large file from the beginning, back when it was just the tile engine for a Terraria clone. There aren't separate files. It was a headache for sure, but I wanted to avoid function calls across multiple files because they had a (slight) performance overhead and I was trying to squeeze as much performance as possible from the hardware. Developing the engine as separate files and then combining them for release never occurred to me. 

 

You have my blessing to open source the engine, put it on github, slice it up, whatever you need to do. As long as the engine itself remains free to use.

[TOPIC CONTROLS]
This topic has been archived. This means that you cannot reply to this topic.
[/TOPIC CONTROLS]
[modOptionsDropdown]
[/modOptionsDropdown]
[reputationFilter]
[TOPIC: post.html]
#1

johnfl46

[GLOBAL: userInfoPane.html]
johnfl46
  • Observer

  • 8 posts
  • Corona SDK

Hello, I'd like to see about getting the uncompiled version of this library? Id like to setup a source project on github for community driven development and maintenance. Is there another version of this engine that is not in a single large file?



[TOPIC: post.html]
#2

roaminggamer

[GLOBAL: userInfoPane.html]
roaminggamer
  • Corona Geek

  • 7,641 posts
  • Corona SDK

MTE isn't compiled, it is just a single source file: mte.lua

 

Did you get your copy from the link here?  https://forums.coronalabs.com/topic/33119-million-tile-engine-beta-release/ 

 

Have you tried contacting the author (dyson122) about this?

 

 

Anyways, the latest/last-released version is: MTE 0.990.  It comes in a zip/rar file with the module mte.lua, docs, examples, and the prior version in a sub-folder with appropriate older docs and samples.



[TOPIC: post.html]
#3

AppDeveloperGuy

[GLOBAL: userInfoPane.html]
AppDeveloperGuy
  • Enthusiast

  • 36 posts
  • Corona SDK

Hello, I'd like to see about getting the uncompiled version of this library? Id like to setup a source project on github for community driven development and maintenance. Is there another version of this engine that is not in a single large file?

 

Hi, johnfl46. roaminggamer is correct. MTE is a fantastic engine, performance and feature wise (with great documentation), but I agree the code would certainly benefit from being completely refactored and organised into multiple files (classes) for easier maintenance and future (community) development. MTE would also benefit greatly from implementing middleclass.lua to provide OOP support (inheritance, polymorphism, etc) - as would most lua code TBH.

 

I'd be very interested in contributing to the project. I wonder whether others here would be too?

 

Have you contacted dyson, the author of MTE, with regards to licensing issues? An MIT License for the project (and his blessing) would be ideal.



[TOPIC: post.html]
#4

johnfl46

[GLOBAL: userInfoPane.html]
johnfl46
  • Observer

  • 8 posts
  • Corona SDK

In response to both posts.

 

MTE isn't compiled, it is just a single source file: mte.lua

 

That is true. When I think of compiling with scripting languages, I think of concatenating all files into a single file and optionally minifying/obfuscation of the code. The various files would be placed in a single file in the correct order so classes, etc would be loaded when it should be.

 

I'm thinking the author has the source code split up into folders and files and when he released the library, it all got put into one file for easier loading. I could be wrong though but developing a library in a 22k ish line file would be a headache.

 

Have you contacted dyson, the author of MTE, with regards to licensing issues? An MIT License for the project (and his blessing) would be ideal.

 

I have not yet. When I was searching I couldn't find a way to message him. (I was probably just not logged in ^_^) I'll send dyson a message now and link to this topic.

 

This is something worth open sourcing after my experience with it these past couple months.



[TOPIC: post.html]
#5

dyson122

[GLOBAL: userInfoPane.html]
dyson122
  • Contributor

  • 590 posts
  • Corona SDK

  Best Answer

I developed MTE as a single large file from the beginning, back when it was just the tile engine for a Terraria clone. There aren't separate files. It was a headache for sure, but I wanted to avoid function calls across multiple files because they had a (slight) performance overhead and I was trying to squeeze as much performance as possible from the hardware. Developing the engine as separate files and then combining them for release never occurred to me. 

 

You have my blessing to open source the engine, put it on github, slice it up, whatever you need to do. As long as the engine itself remains free to use.



[TOPIC: post.html]
#6

AppDeveloperGuy

[GLOBAL: userInfoPane.html]
AppDeveloperGuy
  • Enthusiast

  • 36 posts
  • Corona SDK

I developed MTE as a single large file from the beginning, back when it was just the tile engine for a Terraria clone. There aren't separate files. It was a headache for sure, but I wanted to avoid function calls across multiple files because they had a (slight) performance overhead and I was trying to squeeze as much performance as possible from the hardware. Developing the engine as separate files and then combining them for release never occurred to me. 

 

You have my blessing to open source the engine, put it on github, slice it up, whatever you need to do. As long as the engine itself remains free to use.

 

This is excellent news, thank you, dyson122.

 

MTE is a great engine and I'd love to see it grow and thrive through active community-driven development, continuing your good work! :)

 

johnfl46, what are your thoughts / plans from here? As I say, I'd be very interested in contributing to the project.

 

Please feel free to PM me to discuss further or we can continue the conversation on this thread; giving other developers the opportunity to contribute and shape the future of MTE.



[TOPIC: post.html]
#7

johnfl46

[GLOBAL: userInfoPane.html]
johnfl46
  • Observer

  • 8 posts
  • Corona SDK

This is excellent news, thank you, dyson122.

 

MTE is a great engine and I'd love to see it grow and thrive through active community-driven development, continuing your good work! :)

 

johnfl46, what are your thoughts / plans from here? As I say, I'd be very interested in contributing to the project.

 

Please feel free to PM me to discuss further or we can continue the conversation on this thread; giving other developers the opportunity to contribute and shape the future of MTE.

 

 

Sorry for the delay! This has been on my todo list for a bit. Glad to say we are now online and ready to go!

https://github.com/jsykes/million-tile-engine

 

As for plans.. I think the first and highest priority is to get the source code split up into multiple files and organized by some type of folder/file architecture. Then we can use gulp, grunt, or some lua concat/compiler to output to the dist folder.

 

Beyond that is to hammer out any bugs or issues that come up. We are starting at version 0.9.9 like it originally was. I think 1.0 is good after we finish these couple things.

 

We can keep chatting on here or use PM's. Could also probably collaborate on the GitHub repo. I'm new to hosting something that others contribute to but it shouldn't be too crazy.

 

Also, the repo has the first release already which is a snapshot of what dyson122 released.



[TOPIC: post.html]
#8

AppDeveloperGuy

[GLOBAL: userInfoPane.html]
AppDeveloperGuy
  • Enthusiast

  • 36 posts
  • Corona SDK

Sorry for the delay! This has been on my todo list for a bit. Glad to say we are now online and ready to go!

https://github.com/jsykes/million-tile-engine

 

As for plans.. I think the first and highest priority is to get the source code split up into multiple files and organized by some type of folder/file architecture. Then we can use gulp, grunt, or some lua concat/compiler to output to the dist folder.

 

Beyond that is to hammer out any bugs or issues that come up. We are starting at version 0.9.9 like it originally was. I think 1.0 is good after we finish these couple things.

 

We can keep chatting on here or use PM's. Could also probably collaborate on the GitHub repo. I'm new to hosting something that others contribute to but it shouldn't be too crazy.

 

Also, the repo has the first release already which is a snapshot of what dyson122 released.

 

Hi John, good work and glad to hear back. I was going to PM you this morning actually (UK time) so this is pretty timely!

I have some good news: I've already started. I spent a good chunk of this weekend going through the mte.lua ( 0v990-1 ) file and moving functions out in to separate modules. At 17,400 lines of code it took a while and there's still 13,000 more to go.

Going through the source you can see that MTE is clearly a labour of love as well as a living work-in-progress. I've found some code duplication that I've cleaned up - for example levelToLoc, levelToLocX and levelToLocY were essentially the same algorithm copied and amended three times - now levelToLocX and levelToLocY just call levelToLoc and return the value you need. This reduced a few lines and will make it easier to read/maintain. The same went for levelToScreenPos, screenToLoc and those other similar functions. The functions that read and parse XML were also included twice in two separate places (I checked and they were identical) and now that's just the one being called. I dare say we'll find other opportunities for refactoring and optimisation as we go through as well as the need to create some new private functions to perform reoccurring get/set/processing tasks.

Essentially this is just a starting point and the modules I've created - Map, Camera, Light, Xml, etc - are just simple namespace tables that have begun to separate the various concerns with a view to eventual full encapsulation. They aren't perfect yet and will require further refactoring as there are some cross-references in there which have prevented me from moving other functions... but it's a start. Once we can see how it's all pieced together it should be easier to make those design decisions about what really should go where.

So I agree. I think phase one is to split it all up broadly so it's easier to work with initially. As we go, look for opportunities for duplication removal, refactoring and optimisation so it's easier to read, understand and manage. Once that's done we'll be ready to organise properly in to an architecture with distinct classes that will be the foundation of MTE version 1.0.

Finally, all the work I've done so far provide non-breaking changes with the mte.lua file still exposing the same public functions as before (e.g. M.levelToLoc = Map.levelToLoc ). I've also moved all the demo projects in to a single folder called "demos" which all now reference the exact same MTE file and share the same build settings, config and main.lua. Trying the various samples and testing the MTE file (for breaking changes following refactoring) is now a trivial case of commenting/uncommenting the require functions to each demo in the main.lua file. Corona's new Live Builds has made on-device testing of all these sample projects in a single app an absolute breeze too. I love Live Builds!

I'm quite happy to keep chatting openly on here if you are - broadly, at least for now - as it'll show the community that MTE is back in active development and will indicate its current status. But of course PMs are welcome too as will be necessary to discuss the finer points of development and there's no need to pollute the public forum with that tedium. ;)

All the best,

Paul



[TOPIC: post.html]
#9

johnfl46

[GLOBAL: userInfoPane.html]
johnfl46
  • Observer

  • 8 posts
  • Corona SDK

Hi John, good work and glad to hear back. I was going to PM you this morning actually (UK time) so this is pretty timely!

I have some good news: I've already started. I spent a good chunk of this weekend going through the mte.lua ( 0v990-1 ) file and moving functions out in to separate modules. At 17,400 lines of code it took a while and there's still 13,000 more to go.

Going through the source you can see that MTE is clearly a labour of love as well as a living work-in-progress. I've found some code duplication that I've cleaned up - for example levelToLoc, levelToLocX and levelToLocY were essentially the same algorithm copied and amended three times - now levelToLocX and levelToLocY just call levelToLoc and return the value you need. This reduced a few lines and will make it easier to read/maintain. The same went for levelToScreenPos, screenToLoc and those other similar functions. The functions that read and parse XML were also included twice in two separate places (I checked and they were identical) and now that's just the one being called. I dare say we'll find other opportunities for refactoring and optimisation as we go through as well as the need to create some new private functions to perform reoccurring get/set/processing tasks.

Essentially this is just a starting point and the modules I've created - Map, Camera, Light, Xml, etc - are just simple namespace tables that have begun to separate the various concerns with a view to eventual full encapsulation. They aren't perfect yet and will require further refactoring as there are some cross-references in there which have prevented me from moving other functions... but it's a start. Once we can see how it's all pieced together it should be easier to make those design decisions about what really should go where.

So I agree. I think phase one is to split it all up broadly so it's easier to work with initially. As we go, look for opportunities for duplication removal, refactoring and optimisation so it's easier to read, understand and manage. Once that's done we'll be ready to organise properly in to an architecture with distinct classes that will be the foundation of MTE version 1.0.

Finally, all the work I've done so far provide non-breaking changes with the mte.lua file still exposing the same public functions as before (e.g. M.levelToLoc = Map.levelToLoc ). I've also moved all the demo projects in to a single folder called "demos" which all now reference the exact same MTE file and share the same build settings, config and main.lua. Trying the various samples and testing the MTE file (for breaking changes following refactoring) is now a trivial case of commenting/uncommenting the require functions to each demo in the main.lua file. Corona's new Live Builds has made on-device testing of all these sample projects in a single app an absolute breeze too. I love Live Builds!

I'm quite happy to keep chatting openly on here if you are - broadly, at least for now - as it'll show the community that MTE is back in active development and will indicate its current status. But of course PMs are welcome too as will be necessary to discuss the finer points of development and there's no need to pollute the public forum with that tedium. ;)

All the best,

Paul

 

Wow! That is awesome. I wasn't expecting anything to have been started just yet. So you should be able to clone the repo down and make a refactoring branch to add what you've done into it. The src folder should do well for all the split up files. The demo stuff sounds good. Were all the build settings and configs for the demos the same? We can use those samples for testing the library to make sure it remains backwards compatible. I'll start looking into methods of concatenating the various files. Also, if you end up changing folder names, just give the README an update.. it's still in the early stages though.



[TOPIC: post.html]
#10

StarCrunch

[GLOBAL: userInfoPane.html]
StarCrunch
  • Contributor

  • 842 posts
  • Corona SDK

I'll start looking into methods of concatenating the various files.

 

"Lua amalgamation" turns up a few entries, including this nicely annotated one.



[TOPIC: post.html]
#11

AppDeveloperGuy

[GLOBAL: userInfoPane.html]
AppDeveloperGuy
  • Enthusiast

  • 36 posts
  • Corona SDK

"Lua amalgamation" turns up a few entries, including this nicely annotated one.

 

That's great, thanks StarCrunch. The link above also identifies a few other options, which I've listed below along with Amalg.

 

Amalg : https://siffiejoe.github.io/lua-amalg/

Squish : http://matthewwild.co.uk/projects/squish/home

LOOP : http://loop.luaforge.net/release/preload.html

bundle: https://github.com/akavel/scissors/blob/master/tools/bundle/bundle.lua



[TOPIC: post.html]
#12

AppDeveloperGuy

[GLOBAL: userInfoPane.html]
AppDeveloperGuy
  • Enthusiast

  • 36 posts
  • Corona SDK

Wow! That is awesome. I wasn't expecting anything to have been started just yet. So you should be able to clone the repo down and make a refactoring branch to add what you've done into it. The src folder should do well for all the split up files. The demo stuff sounds good. Were all the build settings and configs for the demos the same? We can use those samples for testing the library to make sure it remains backwards compatible. I'll start looking into methods of concatenating the various files. Also, if you end up changing folder names, just give the README an update.. it's still in the early stages though.

 

Will do. I'll take a look at this at the weekend if I can. I'm afraid I won't get time again until then.

 

The build settings and configs for the demos were not all the same but didn't need to be different so I just made the necessary changes to each one.



[TOPIC: post.html]
#13

johnfl46

[GLOBAL: userInfoPane.html]
johnfl46
  • Observer

  • 8 posts
  • Corona SDK

Will do. I'll take a look at this at the weekend if I can. I'm afraid I won't get time again until then.

 

The build settings and configs for the demos were not all the same but didn't need to be different so I just made the necessary changes to each one.

 

No problem, thanks for contributing!

Give the repo a check. I've added amalg and updated the README to show the command used to compile the src directory into the dist/mte.lua bundle. It looks promising.

 

Also, I had a small edit on the mte I was using to fix a bug with loadMap not using the passed in baseDirectory value. This is on line 9803 and I changed it to

 

local tempSet = xml.loadFile(directory..temp.child[i].properties.source, base)

 

I'll hold off on any more edits to mte until we merge things together again. If you can toss in that line in your version, it should merge well.



[TOPIC: post.html]
#14

AppDeveloperGuy

[GLOBAL: userInfoPane.html]
AppDeveloperGuy
  • Enthusiast

  • 36 posts
  • Corona SDK

No problem, thanks for contributing!

Give the repo a check. I've added amalg and updated the README to show the command used to compile the src directory into the dist/mte.lua bundle. It looks promising.

 

Also, I had a small edit on the mte I was using to fix a bug with loadMap not using the passed in baseDirectory value. This is on line 9803 and I changed it to

 

local tempSet = xml.loadFile(directory..temp.child[i].properties.source, base)

 

I'll hold off on any more edits to mte until we merge things together again. If you can toss in that line in your version, it should merge well.

 

All done and pull request made!

 

I've not had time to look at amalg but hopefully the changes I've made will work with it.

 

I should note I've introduced a minor issue with the Sledge sample project: The red sledge image has accidentally gone off-camera. This occurred after moving cameraX and cameraY variables to the Camera module and it needs fixing. Again, not had time to look at it today but the sledge shadow is still there at least. ;)



[TOPIC: post.html]
#15

johnfl46

[GLOBAL: userInfoPane.html]
johnfl46
  • Observer

  • 8 posts
  • Corona SDK

All done and pull request made!

 

I've not had time to look at amalg but hopefully the changes I've made will work with it.

 

I should note I've introduced a minor issue with the Sledge sample project: The red sledge image has accidentally gone off-camera. This occurred after moving cameraX and cameraY variables to the Camera module and it needs fixing. Again, not had time to look at it today but the sledge shadow is still there at least. ;)

 

Merged! It looks great so far, pretty well organized.

 

I updated the readme to show amalg's new usage. It adds the -c flag for using the amalg.cache file. This file is what we will use to list (in order) our module dependencies. So long as we keep some sort of main.lua (or mte.lua) file that we feed to amalg.. it should toss it all together. The current code works with amalg now and my side project loads up with it.

 

I did notice that calling mte.getObjectLayer(1) with the new compiled output leads to Attempt to call field 'getObjectLayer' (a nil value)

 

We may need to feed these separate modules into the local M = {} object so that mte object's self still contains these separate modules as an extension. I'm hesitant to poke around and commit just yet until you feel we are good to start fixing bugs. Also, if you want me to lend a hand at refactoring, also let me know. You're doing well on your own but 13k remaining lines is quite the undertaking.

 

I'll put this error up as an issue so I can be reminded to address it at some point. 



[TOPIC: post.html]
#16

AppDeveloperGuy

[GLOBAL: userInfoPane.html]
AppDeveloperGuy
  • Enthusiast

  • 36 posts
  • Corona SDK


I did notice that calling mte.getObjectLayer(1) with the new compiled output leads to Attempt to call field 'getObjectLayer' (a nil value)

 

We may need to feed these separate modules into the local M = {} object so that mte object's self still contains these separate modules as an extension. I'm hesitant to poke around and commit just yet until you feel we are good to start fixing bugs. Also, if you want me to lend a hand at refactoring, also let me know. You're doing well on your own but 13k remaining lines is quite the undertaking.

 

I'll put this error up as an issue so I can be reminded to address it at some point. 

 

Fixed and pr made. Yes, I missed that one off accidentally - I mean, just checking you were paying attention. Hopefully it's the only one but feel free to fix if you spot any others. :)

 

Those changes are now everything I've done so we're up to date. Please do go ahead and join in with the refactoring - as you say, there's a bit of work ahead. I'll only really get chance to work on this at the weekends (and not even every weekend) so it'll certainly help speed things along. Also, time zone differences may help protect against duplication of effort; so as long as we commit and push as we go everything should be seamless.

 

I guess the next step is to target those other smaller functions that could easily be moved out of the main MTE file leaving just the really long ones behind. Those will need to be broken down further but it should be easier to achieve with fewer lines of code in the main MTE file to work around.



[TOPIC: post.html]
#17

johnfl46

[GLOBAL: userInfoPane.html]
johnfl46
  • Observer

  • 8 posts
  • Corona SDK

I've made some headway on refactoring. It took a while to get things back up and running as I've taken out a few thousand line update function among some others. I've put them into a Core module but this file is getting large as well. We will need to add a Core folder and split it up even further at some point. I've also come across functions that are in and outside of core that may be useful to be inside of a Render module or something.

 

Anyway, master has the latest.. the samples load up and the only thing that really stood out was the lighting was missing on the lighting sample. I must have broke the lightbulb =]

 

Also, what do you think about conventions? I have a habit of adding a ; at the end of lines. I can leave it off since the source does not use them. Also for lua file names, are you fine with capital first letter files or all lowercase? I'm used to lowercase for scripting and uppercase for managed languages like c#. But it's really a non issue for me.




[topic_controls]
[/topic_controls]