... (WIP)



====== Icedove-UXP =======

{{ :icedove_icon.png?200|}}

Icedove-UXP is a mail application built on the the [[https://github.com/MoonchildProductions/UXP|Unified XUL Platform]].

Historically, Icedove was a re-branding of Thunderbird by the Debian project, with minimal modifications, in order to resolve branding disputes. Debian since [[http://fossforce.com/2017/02/great-debian-iceweasel-icedove-saga/|discontinued Icedove]] in February of 2017 after reaching an agreement with Mozilla over the use of it's trademark.

The Hyperbola Team continued the Icedove branding in the spirit of cultural freedom for several months. However, inconsistencies were discovered, including the use of non-free fonts in Debian's Icedove logo. Since Debian no longer supported Icedove and was encouraging users to install Thunderbird with the new Quantum Engine, we decided to look into creating a proper fork. 

Icedove was the second application to be successfully ported to UXP by Hyperbola Project.

===== Application Info =====

  * Latest tarball: [[https://repo.hyperbola.info:50000/other/icedove-uxp/icedove-uxp-2.0.tar.gz|v2.0]] [[https://repo.hyperbola.info:50000/other/icedove-uxp/icedove-uxp-2.0.tar.gz.asc|GPG signature]]
  * Latest commits: [[https://git.hyperbola.info:50100/software/icedove-uxp.git/log/?h=v2.0|v2.0 tag]]
  * Older tarballs: [[https://repo.hyperbola.info:50000/other/icedove-uxp/|since v2.0]]
  * Source code: [[https://git.hyperbola.info:50100/software/icedove-uxp.git/|https]]
  * Version: v2.0
  * Resources:
    * [[https://issues.hyperbola.info/|Bug Tracker]]
    * [[https://git.hyperbola.info:50100/software/icedove-uxp.git/plain/CHANGELOG.md|Change Log]]
===== Porting Notes =====
In order to preserve the knowledge of porting an application to UXP application for future developers, this is step-by-step guide  to how Icedove was ported to the Unified XUL Framework.

=== Get the source and setup the build environment ===

As with any project, the first thing you should do is start from the source. In this case any XUL application < 52.x should work. 
Versions 60.x and higher use the so-called Quantum engine which guts XUL in favor of Chromium dependencies.

To setup the build environment, you will need to install some dependencies. 
The package names may vary slightly depending on your GNU/Linux distribution. You can also refer to [[https://web.archive.org/web/20180729032348/http://www.linuxfromscratch.org/blfs/view/svn/xsoft/thunderbird.html|LFS]] notes.

<code>pacman -S alsa-lib dbus-glib ffmpeg gtk2 gtk3 hunspell icu libvpx libxt mime-types mozilla-common nss sqlite startup-notification ttf-font
autoconf-legacy clang diffutils gconf git imake inetutils libpulse llvm mesa pkg-config python2 quilt unzip xorg-server-xvfb yasm zip</code>

We started with Thunderbird 52.9.1 as it was the latest stable release at the time of this writing.
The original sources can be found here: http://http.debian.net/debian/pool/main/t/thunderbird/thunderbird_52.9.1.orig.tar.xz

<code>tar -jXf thunderbird_52.9.1.orig.tar.xz -c ~/Projects/mailclient</code>

Once you've extracted the sources, the first thing you should do is delete the /mozilla folder. 
It contains the core from mozilla-central, which we will be replacing.

<code>cd ~/Projects/mailclient
rm -rf mozilla
</code>

Next, drop in UXP as the replacement:

<code>git clone https://github.com/MoonchildProductions/UXP
mv UXP mozilla
</code>

Now prepare a //mozconfig// for the application and place it into the root folder. //~/Projects/mailclient// in our example.

You may refer to the debug-ready one below:
<code>
# Tell UXP to build the application in the "mail" folder
ac_add_options --enable-application=mail


# Debug settings
ac_add_options --disable-optimize
ac_add_options --enable-debug-symbols
ac_add_options --enable-debug

ac_add_options --with-pthreads

# Target platform
ac_add_options --target=x86_64-pc-linux-gnu

# Specify number of CPU Cores
mk_add_options MOZ_MAKE_FLAGS="-j2"

# Disable bloatware that might be in the application
ac_add_options --disable-accessibility
ac_add_options --disable-alsa
ac_add_options --disable-cpp-rtti
ac_add_options --disable-crashreporter
ac_add_options --disable-dbus
ac_add_options --disable-necko-wifi
ac_add_options --disable-negotiateauth
ac_add_options --disable-official-branding
ac_add_options --disable-pulseaudio
ac_add_options --disable-tests
ac_add_options --disable-updater
ac_add_options --disable-webrtc
ac_add_options --disable-safe-browsing
ac_add_options --disable-url-classifier

# Enable some useful features

# Lightning Calendar
ac_add_options --enable-calendar
# Gnome GIO
ac_add_options --enable-gio

# System Libs confirmed working:
ac_add_options --with-system-icu
ac_add_options --with-system-jpeg
ac_add_options --with-system-zlib
ac_add_options --with-system-bz2
ac_add_options --with-system-libvpx
ac_add_options --enable-system-hunspell
ac_add_options --enable-system-ffi
ac_add_options --enable-system-pixman
#ac_add_options --enable-system-sqlite # keep disabled for stability
#ac_add_options --with-system-nspr # keep disabled for stability
#ac_add_options --with-system-nss # keep disabled for stability
</code>

From here you should familiarize yourself with [[https://web.archive.org/web/20180729033524/https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/mach|mach build]] as it will help you a lot when you begin to build and debug your application. You can test "//./mozilla/mach build//" at this point, but it will not complete properly until we have fixed a number of build errors.

=== Missing files ===
Now that your build environment is ready, we need to include files which are missing from your application. When you downloaded the original sources, Mozilla neglected to tell you that they were incomplete.
You will need to find and install missing files. 

  * All XUL applications need an [[https://git.hyperbola.info:50100/software/icedove-uxp.git/plain/mail/app/application.ini?id=fbe68f2140971e7c6f9672af4ab21af559d3f01d|application.ini]], which is not included. The global variable @MOZ_APP_VERSION@ comes from confvars.sh, whereas @GRE_MILESTONE@ comes from UXP Toolkit.

  * In our case, the program depended on [[https://git.hyperbola.info:50100/software/icedove-uxp.git/commit/?id=bc4bf14c66ffc6780238646ecf51022e9e60a840|Twemoji font]] which was not included. We added it to the application and changed the path to a local one as seen in the commit.

=== Removed files ===
UXP is known for removing bloatware. As such, you will spend a fair amount of time removing bloat from your application which is no longer supported by UXP.

  * UXP does not use Google Phishing/nsURLClassifier, so it should be disabled in mozconfig and removed from [[https://git.hyperbola.info:50100/software/icedove-uxp.git/commit/?id=0bc4bf5744e67a5481e8483ad5572b5e2d8df147|installer/package-manifest.in]].

  * UXP does not use "dom_webspeechrecognition.xpt" or "dom_webspeechsynth.xpt". You need to remove or comment these out in [[https://git.hyperbola.info:50100/software/icedove-uxp.git/commit/?id=3af07e2d38bed38b046d6207d4946a2db9aa63e9|package-manifest.in]].

  * UXP does not use Security Reporter. You need to remove or comment these out in [[https://git.hyperbola.info:50100/software/icedove-uxp.git/commit/?id=272272af52046eb8e11dcd1b99f503c6b03a9716|package-manifest.in]].

  * There is no support for Rust. If your application has Rust dependencies (MOZ_RUST_URLPARSE), they must be removed. This gets a little more complex, so it helps to spend some time going through commits that your application made during the transition to Rust and revert them. In our case, we were lucky to find the [[https://hg.mozilla.org/comm-central/log?rev=1317724|initial rust commit]] and revert it.

  * There is no support for Chromium MOZ_SANDBOX. You will need to remove it from your application. Although you can go back find the commits that added this, once you know the pattern of "XRE_XPCShellMain" is fairly easy to remove from any application. It is usually inside ifdef blocks. You can view our removal [[https://git.hyperbola.info:50100/software/icedove-uxp.git/commit/?id=7280ac014c7e812c78d7fa78d24794f2d45463dc|here]].

=== Fixing API changes ===

  * UXP does not use nsIURIWithQuery (neither does Mozilla, so you shouldn't have this in many non-legacy applications). We had to backport the removal from [[https://hg.mozilla.org/comm-central/log?rev=1326433|comm-central 1326433]] as the source we were using did not have this patch yet.

  * UXP no longer allows SEC_NORMAL for security reasons (neither does Mozilla, so you shouldn't have this in many non-legacy applications). If this is unresolved, your application will build - but you will get complex segfaults in your application with strange and uninformative errors. Finding this bug took 3 days of searching and gdb usage, so we are happy to inform you of this issue so that you do not suffer the same fate. Backporting [[https://hg.mozilla.org/comm-central/log?rev=1328847|comm-central 1328847]] solved this issue.

  * UXP does support Error Console, whereas Mozilla does not. We [[https://git.hyperbola.info:50100/software/icedove-uxp.git/commit/?id=6ac15cada7c74bce45bb921a167d09dac4f4f3a5|added Error Console back]] to our application to help with debugging.

  * UXP's addon manager uses a unified toolkit module. You have to [[https://git.hyperbola.info:50100/software/icedove-uxp.git/commit/?id=b3aaf19f4ef6192f8d20c074d37c5dae651abcea|include the blocklist.manifest inside package-manifest.in]] for the TychoAM to work properly. Otherwise, the addons page will fail with XPCU_serviceLambda@XPCOMUtils.jsm:230:7.


By this point your application should be building properly, and you can launch it with "//./mozilla/mach run//", but there are still a number of bugs remaining.


=== Tidying up the layout ===

  * Our application did not reference the close tab CSS, which caused there to be no close button. [[https://git.hyperbola.info:50100/software/icedove-uxp.git/commit/?id=1d7862d29bb00b1c2580cf9f0e9cc372d5a8c3aa|We added it to the application]]. Depending on your application, you may need to use MOZ_AUSTRALIS=1 in confvars.sh to get the proper CSS for this to work. We did not need it for this application, but it was needed in [[Iceweasel-UXP]].

  * UXP has a strange CSS issue related to the findBar. If a word is not found it will paint the window red as it should, but also create a large whiteblock overtop the text, which causes frustration for findBar users. This appears to only affect GNU/Linux builds, so we [[https://git.hyperbola.info:50100/software/icedove-uxp.git/commit/?id=e89c421013a42ec88248759b49a7a7e20cf3cc87|override the chrome file and css]] in our application.

=== Making it yours ===
The rest of your job will be on branding. You should come up with a good name for your new application, and a proper logo. 

Branding resides in the ///branding// folder respectively. You may wish to modify the images with GIMP to create your brand.

Once you have chosen a good name, it's time to go through the [[https://git.hyperbola.info:50100/software/icedove-uxp.git/commit/?id=059972281180442936f6b03e407e2658de662f6a|entire codebase]] to change the application name. You may also want to rename the [[https://git.hyperbola.info:50100/software/icedove-uxp.git/commit/?id=18826725186f45b1fcd23a1956585f15dc635fd2|preference js files]] for branding purposes.

You may wish to change the build structure. Although we did not do it for this application, you may refer to [[https://github.com/MoonchildProductions/UXP/issues/303|UXP's commits]] on moving a browser application and the respective path changes throughout the application.

We also removed [[https://git.hyperbola.info:50100/software/icedove-uxp.git/diff/calendar/providers/caldav/calDavCalendar.js?id=0ddac1d1a353399bdb08ae9ec9ad44441d68007a|proprietary API keys and obfuscated JavaScript]] we discovered in the application. //[[https://bugzilla.mozilla.org/show_bug.cgi?id=849540#c13|Email != Web. We cannot allow email to depend on the web, or proprietary auth mechanisms.]]//

As a final measure, you may wish to [[https://git.hyperbola.info:50100/software/icedove-uxp.git/commit/?id=4286f8817b32ab5afa377d4d208058bf0aecc0ec|change the GUID]]. Although this breaks extension compatibility from the application you forked, it allows for much more customization later on.

At this point, there will be trivial bugs to fix, or features you may wish to create or disable. The choice is up to you.