Tryton News: Tryton Release 5.2

@ced wrote:

We are proud to announce the 5.2 release of Tryton. This is the first minor release which means that it will be supported for 1 year only.
As usual the migration from previous series is fully supported. Some manual operation may be required, see Migration from 5.0 to 5.2.

This release will be presented at the Tryton – Unconference Marseille – June 4th-7th, 2019

Here is the list of the most noticeable changes:
(For a more complete list, see the change log of each package)

Contents

Changes For The User

We have a new widget which allows to edit HTML content using a WYSIWYG editor. The widget is available in both clients. It can be used for example to edit a product description field for a web shop site.

We have reworked the CSV import/export to be more user-friendly. For example, the saved exports are now available directly under the print toolbar like if it was a report.
We also use by default the locale configuration to format the data (e.g. date and numbers). This provides a better operating system integration with the used programs.
The CSV import can now update existing records if their ID is provided.

The calendar view (which allows to display records on a calendar) already had a month and week view. Now it has also a day view thanks to Release of GooCalendar 0.5.

We improved the visibility of the notes and attachments. There is now a different color between unread and read notes. We display also the total of notes (instead of only the unread). The desktop client now has a badge to warn about the presence of notes or attachments, when the toolbar only shows icons.
Desktop resources notification Web resources notification

We have simplified the definition of a scheduled task (also known as cron). The method to run is now a selection (no need to know the internal names). The user selects an interval from minutes to months and can also select the corresponding time. For example to force a task to run every 5 days at :clock230: 2:30am. This is very useful for expensive task that should run when the system is less busy.

We use a range for the number fields on the filter box. The behavior is to create a range between the two values if they are different. And by default when the first part is modified, it update the second to the same value. But when the second part is modified, the first is not. This is the best compromise we found to be the less obstructive to the user but still to provide a powerful option.

When the result of a search is empty and the user has setup an offset, we reduce this offset until there is some result to show. This prevents to confuse the user who may think, there is no result if the offset stays too high.

Searches against codes and numbers have been improved to only match if the search text matches with the starting of the code or number. This is better than matching any parts of the code or number as it is what most users would expect.

One type of error messages that is very difficult to understand for the user, are the access errors. They were very generic and most of the time users could not find a solution by themselves. To improve the situation we show in such error messages, the ids of the record for which the access is forbidden and also the name of the rules that are infringed.

Desktop Client

Now that the desktop client has dropped the support of GTK+2, we can use new widgets from GTK+3 such as the ShortcutsWindow. This window is displayed with the shortcut CTRL+F1 and additionally provides a search functionality.

Another possibility with GTK+3 is to replace the filter popup window by a nicer Popover . This also solves a focus issue that happened on some window managers.

We have been struggling for some releases on providing the best size for the dialogs. We think we have finally found the best solution. The dialog builder searches in the form that it will displayed, if there are any widgets that needs to be expanded. If it found one than the default size will be 150px less than the size of the main window, otherwise the size will be computed from the natural size of each widget.

We missed two shortcuts on One2Many widget to be fully usable with only the keyboard. So we added a shortcut to switch the view and one to remove (and not delete) the selected record.

We added the support of drag & drop on the binary widget (like it already exists on the attachment button).

On the desktop client, we first show the login dialog before the main application window. This has a side effect that it is not possible to know the running version before being connected. As this can lead to some incomprehension if the user is using a wrong client version to connect to a server, we display on the login dialog the client version number.
Desktop login window with version number

The column rendering has been improved which allows now to edit the reference field using a Selection and a Many2One cells and the binary column are using clickable icon cells.

Web Client

The design of the CSV export/import on the web client was not in the best shape. So we put some effort to redesign it to be closer to the desktop client standard.

Also a nice feature of the desktop client is the ability to select a range of records with Shift+Click. This feature was missing on the web client and we added it. Now if you select a first record than click on another one with Shift pressed, the client will select all the records between them.

Another missing feature from the desktop client is now implemented: the ability to create attachment by drag & drop on the toolbar button.

In order to improve the navigation between tabs in the web client, we allow to use Alt+Tab to switch between them.

The URL in the browser can be shared between users to open the same view. But it was missing the definition of the “tab domain”. This is fixed now. If the view has tabs, the URL will contain their definition and other users will see them also.

Accounting

To simplify the account creation, we merged the two concepts of type and kind. Now everything is defined on the type. This is simpler for the user because now an account only requires a name, a parent and a type (inherited by default from the parent). Also new, a type can have multiple usage like revenue and expense etc.
All the standard charts of accounts have been updated to this new format.

Sometimes you want to deduct from a supplier invoice and a credit note from the same supplier. We have added a wizard that groups lines from the same party and leave the remaining to the right account. All the grouped lines are reconciled together but delegates their status to the remaining line. So the invoices and credit notes will be marked only once the remaining line will be reconciled also.
Only one payment needs to be created for the remaining line.
The payment wizard warns about if the selected lines could be grouped with others before creating the payments.

Once an invoice is paid, the existing payment lines are replaced by the list of the lines used for the reconciliations. This shows to the user the exact way the invoice was paid.

When an invoice is refunded by a credit note using the credit wizard, its state is set to cancel instead of paid. This gives a better picture of the reality.

Spanish

The report 303 has been updated to the latest version and it displays the amount to compensate from the previous periods. The new template can be applied on older versions.
We added two more reports for Spain:

  • The VAT listing with the Spanish codes
  • The EC operation list

French

The accounts 400 and 410 has been changed into view as they must contain the sum of respectively all accounts starting by 40 and those starting by 41.

Analytic

We added the product and category criteria to the analytic rule engine. This allows to write more complex automatic rules.
Now those rules are only applied to the income statement lines as it is a most expected usage of analytic accounting.
Since the introduction of the rule engine, it was clear that the mandatory flags on analytic roots were no more useful (even annoying). So we removed them.

Notification

In addition to the users and party, we can now use an employee field as recipient of a notification.

Party

Until now, we had only the European VAT identifier by default. But Tryton aims to be usable all around the world. So we added all the party identifier available in python-stdnum library and we flagged those that can be used as tax identifier.
If you are missing an identifier, we encourage you to contribute it to python-stdnum so it will be available in the next release.

Sometimes, users encode phone number without any country prefix. Until now, this raised a validation error. Now Tryton tries to find a country from the party that works. By default it tries all the countries from the addresses.

Purchase

Until now, the product supplier definition was only available per product but not per variant. This is corrected now. The product suppliers defined on the variant are used first and if none match the criteria (e.g. the right supplier), the product suppliers are searched.

Stock

We found that the picking algorithm may choose a children location, even if there were products available in the source location. This was astonishing for the user, so we decided to pick first in the source location before the children locations.

Now we allow to consume products from a supplier consignment directly to a customer. Tryton will create the corresponding supplier invoice line and the customer invoice line.

Until now a unit price was set by default when a stock move was created manually. We stopped this behavior because the proposed unit price was most of the time wrong. So we prefer that the user actually set it if it is required as it will be more accurate.

New Modules

Account Statement Rule

The account_statement_rule module allows to define rules to automatically complete statement lines from imported files.

Marketing Automation

The marketing_automation module allows marketing actions to be automated. It is based on scenarios and activities that are executed on selected records.

Sale Product Customer

The sale_product_customer module defines customer’s names and codes for products and/or variants. A reference to the customer’s product is added to the sale line.

User Role

The user_role module allows to assign roles to user instead of groups. A Role is defined by a set of groups. When a role is added to a user, it overrides the existing groups. A role can be added to a user for a period.

Web Shortener

The web_shortener module allows URLs to be shortened. It counts the number of times the URL is accessed and optionally triggers action.

New Languages

This release receive translations for those new languages: Estonian, Turkish and Finnish.
If you want to add your language or improve existing one, see how to contribute to the translations.

Changes For The Developer

In this release, we remove the support of Python 3.4 which has reached its end-of-life the last March.

A new type of view has landed in Tryton. It is the list-form which is a combination of the list and the form views. It displays on a list an editable form for each record. This gives more control on the layout than the standard list view but it is less performant on large sets of records.

We added a new widget HTML. This widget allow the user to edit HTML content using a WYSIWYG editor (we choose TinyMCE ). The editor plugins, CSS and class can be customized per model and field and it support translated fields.
This widget can be used for example to edit a product description which will be used on a web-shop. In this case, Tryton can be configured to reuse the CSS file from the web-shop to show the same rendering.
The widget is also used to edit HTML report template like for the emails.

We added a console to the server. This console starts within a transaction and provide access to all objects. The transaction must be explicitly commit otherwise it is roll-backed. The console is by default interactive or it runs the code piped to its standard input.
Such console can be useful when developing to test a function or check data but it can also be useful on production to execute a correction script.

Some efforts have been put on improving the internal cache of Tryton. Now it follow the transaction by storing only committed data and clearing for other transactions once the current transaction is committed.
The synchronization of the caches between different processes (optionally on different hosts) can be configured to use a database channel. In this case the cache is synchronized instantly instead of being based on a timeout.
The cache instances can be configured to automatically invalidate data that stayed in the cache for a specific amount of time. This is useful when storing data that depend on time or on external sources.
Finally, we allowed to control the cache duration of the RPC requests per procedure. The definition of the RPC can have cache attribute which defines the duration to store the result in the cache. The clients use this information and re-use the cached result instead of calling the method as long as the cache is valid. By default, we activated the cache on the common method that have almost static result like ModelView.fields_view_get, ModelView.view_toolbar_get etc. This allowed to reduce by about ⅔ the average number of requests.

By default, the Tryton server uses thread/fork for concurrency. This is not optimal for the long-polling requests that go to the bus. So we added an option to start the server with a corountine model. This is done thanks to the gevent library.
So the common setup is to start a server with thread/fork for the common requests and another one with coroutine for the bus request.

As we could have a second server for the bus request, it is now possible to redirect the bus request to this second server thanks to the url_host configuration. The clients will follow the redirection automatically.
As this redirection can point to a different domain, it is also possible to authorize the cross-origin resource sharing via the configuration.

Sometimes we would like to extend the string or the help text of a field. But the translations did not correctly handle such case because they create an entry for the full text. Now we introduce the concept of partitioned string. It is a Python object that behaves like a string but it keeps the history (parts) of how it was constructed. We use this new object for the string and help text of the field. And we create a translation entry for each part in order to get a modular translation.

We can now search on keys of Dict of fields using the Tryton’s ORM. On PostgreSQL back-end, Dict fields can be stored as a JSON. In this case, the database can use indexes to speed-up the query.
It is also possible to order the search result based on the keys of Dict fields.
The keys with null value are also automatically cleaned.

In order to make Tryton even more modular, the definition of the fields (which is sent to the clients) is now created by the field itself. This allows to create custom fields like the Geometry field of trytond-gis and add new properties like the dimension and geometry_type.

Some times we need to acquire a lock on a record but without modify it (see issue8012). So we added a ModelSQL.lock method which locks the passed records if the database support it or just lock the all table.

Until now, all the errors were raised using the two same exceptions UserError and UserWarning. This was not great for testing nor to catch a specific error.
So we removed the methods raise_user_error and raise_user_warning to replace them by the Python raise statement. This allows to use custom exceptions inheriting from one of the two bases.
As we need to set on the exception the error message for the user, we created a new model to store all the messages ir.message. It can be used with the new gettext tool to get the message in the language of the user.

In order to detect quickly bugs, the Function field raises now a NotImplementedError when we try to set a value but it does not have any setter method.

Until now the ModelStorage.read-API allows to read the field of the model but also the fields of Many2One and Reference targets. The API has been extended to read related xxx2Many fields . Reading a xxx2Many field results in a list of dictionaries with the requested values and always the id . The name of the key is suffixed by a dot, to avoid name space collisions. :warning: The former API for Many2One has been changed to stay consistent.

Some modules have extras dependency. In order to test them, the ModelTestCase has a new attribute extras which is a list that contains the extra modules to activate when running the test. This ensure to cover all the cases.

The domain inversion which allows the client to fill or limit the value of a field based on the existing domain, can now enforce the model of the reference fields.

Desktop Client

In order to prepare the future version 4 of GTK, a effort has been put to remove all the GTK warnings. This required to replace some deprecated widgets like Gtk.Table by Gtk.Grid etc. And finally, we could drop the usage of pygtkcompat, the library for backward compatibility with the version 2 of GTK.

The list views can now have multiple renderers per column. This allowed to edit Reference field with two cells: a selection for the model and a many2one for the record.

Web Client

The desktop client supported plugins since a long time. Now the web client can also be customized with plugins. The default “Translate view” has been implemented as example.

In order to modernize our Javascript code, we started to support defining property on our class implementation. Now the Javascript code is more similar to its Python counterpart which ease the maintenance and avoid bugs.

The web client now supports the expand, height and width attributes. This reduce again the gap between the feature of both clients.

Accounting

Stripe

Fetching the sources from Strip service is quite slow (about 1s per source). So we added a cache of 15’ on the sources of a customer. It is of course cleared if the customer is modified.

Since version 5.0, Tryton has a queue to execute task in background. This queue is now used to charge and capture Stripe payments.

Purchase

As we have already on the purchase request the method find_best_supplier, we added also find_best_product_supplier.

Sale

For performance reason, we changed the fallback unit used by Product.get_sale_price to the sale unit instead of the default unit. This allows services (like a web-shop) to make a single call to retrieve the sale price of all the products without having to make unit conversion.

Following the changes on the sale, the description on sale opportunity is now also optional.

Posts: 1

Participants: 1

Read full topic

Planet Python

Tryton News: Security Releate for issue8189

@ced wrote:

Synopsis

A vulnerability in tryton has been found by Cédric Krier.

With issue8189, an authenticated user can order records based on a field for which he has no access right. This may allow the user to guess values.

Impact

CVSS v3.0 Base Score: 4.3

  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: Low
  • User Interaction: None
  • Scope: Unchanged
  • Confidentiality: Low
  • Integrity: None
  • Availability: None

Workaround

There are no known workarounds.

Resolution

All affected users should upgrade trytond to the latest version.
Affected versions per series:

  • 5.0: <=5.0.5
  • 4.8: <=4.8.9
  • 4.6: <=4.6.13
  • 4.4: <=4.4.18
  • 4.2: <=4.2.20

Non affected versions per series:

  • 5.0: >=5.0.6
  • 4.8: >=4.8.10
  • 4.6: >=4.6.14
  • 4.4: >=4.4.19
  • 4.2: >=4.2.21

Reference

Concern?

Any security concerns should be reported on the bug-tracker at
https://bugs.tryton.org/ with the type security .

Posts: 1

Participants: 1

Read full topic

Planet Python

Tryton News: Newsletter March 2019

@ced wrote:

This month a lot of work has been put on improving and modernize both clients but also on increasing the maintainability of the code.
We also want to remind you that the registration for Tryton Unconference at Marseille, the 4th-7th June is opened. Do not wait too much, the places are limited.

Contents:

Changes For The User

The AEAT303 Spanish report has been updated to follow the new format published this year.

The French chart of account has been updated to include the new accounts 442* from PCG 2019.

Now that the desktop client has dropped the support of GTK+2, we can use new widgets from GTK+3 such as the ShortcutsWindow. This window is displayed with the shortcut CTRL+F1 and provide a search functionality.

Another possibility with GTK+3 was to replace the filter popup window by a nicer Popover. This solved also a focus issue that happened on some window managers.

We changed the shortcut to switch tab to CTRL+Tab for desktop and ALT+Tab for web client. This is more natural for the user.

On the desktop client, we show first the login dialog before the main application window. This has a side effect that it is not possible to know the running version before being connected. As this can lead to some incomprehension if the user is using the wrong version to connect to a server, we display on the login dialog the version number.
Desktop login window with version number

The design of the CSV export/import on the web client was not in the best shape. So we put some effort to redesign it to be closer to the Tryton standard.
Before:

After:

On small screen with the web client, replacing search filter may be difficult. So we added a clear button when the input is filled for such case.

On the search filter popover, some kind of fields (e.g. date) are presented as a range. But to enter a equality clause, the user needs to fill both boundary entries with the exact same value. This is tedious. So we changed the range widget to automatically fill the end boundary with the value of the start boundary when this last is changed. It is still possible to create a range query by modifying the to value.

By default Tryton allows only to sale or purchase products that are marked respectively as salable and purchasable. But it happens that over time, we do no want anymore to sale or purchase a product. The problem is that existing orders may no more be valid because of such change. Now we check the flags only on draft and quotation state of the order and existing orders stay valid.

We have remove the mandatory option on analytic axis because it may break some automatic workflow which created document invalid according to the option. As we have since sometimes now a tool to show account lines for which the analytic axis were not completed.

Sometimes it is useful to be able to see quickly the deposit history of a customer. So we added a new relate link from the party form that displays their non-consumed deposit lines.

A frequent reported issue is that the size of dialogs are often too small on the desktop client. We have implemented a new algorithm that provides a better size by default.

We have improved the spacing of the reference field in the web client. Before this change, it could exceed the cell in an editable list.

Changes For The Developer

To be more compliant with HTTP status, the Tryton server will raise 429 TOO MANY REQUESTS when the login rate reach the limit. This gives also a more comprehensive error message to the user.

A request is retried a number of time if it encounters an database operational error. When such error comes from a lock failure which are performed with NO WAIT, the retries are often too fast for the other request to release the lock. So we have added an increasing delay between the retries.

In the last release, we added a Bus to the server which does long polling with the clients. But our main server is based on thread/fork. With big number of users, it can be very resource consuming to keep a thread for each long polling request. So we added an option to run the server using coroutine.
So now, the typical setup for performance is to run a thread/fork server and activate in the configuration the redirection of the bus requests to another Tryton server running with coroutine.

Until now the developer mode of the server (which activate the auto-reload on file changes) activated the debug level of the logging. It was considered annoying so we decoupled the log level from the developer mode. To increase the logging level, you just have to add more -v to the command line (or use a logging configuration).

The Model.fields_get method was a big function with all sort of tests per field type. It was not modular as the fields were hard-coded and it was difficult to maintain. We split the function between each fields which are now responsible of their own definition.
This allowed to provide the dimension and the geometry type for the Tryton GIS backend.

We added the missing support for width and height of the notebook on the web client.

We have modernized the Javascript of the web client and now we use getters and setters. This makes the Javascript code looks closer to the Python code from the desktop client.

We added the support of the window tab in the URL of the clients. This way they do not disappear when the page is reloaded and when the URL is shared.

Posts: 1

Participants: 1

Read full topic

Planet Python

Tryton News: Tryton Unconference 2019: In Marseille on the 6th & 7th of June

@nicoe wrote:

The Tryton Foundation is happy to announce the venue and date of the next Tryton Unconference.

We will go in the sunny city of Marseille in south of France on the 6th and 7th of June. Contrary to previous editions of the Tryton Unconferences the coding sprint will be organized during the two days preceding the conference.

Both events will take place at the École de Commerce et de Management. We will publish a website with more detailed informations shortly.

Many thanks to adiczion which is the organizer of this year event!

Posts: 1

Participants: 1

Read full topic

Planet Python

Tryton News: Newsletter February 2019

@ced wrote:

Tryton continues its road of improvements for more performance and more scaling.

Contents:

Changes For The User

The arrows on columns are now always synchronized with the actual order. If the order is not on a single column then all arrows are displayed.

The records created by XML files in modules are by default protected against modification and deletion. But if they have the attribute noupdate set, they can be modified. Now they can also be deleted and updating the database will not recreate them.

On the wizard that allows to pay multiple lines at once, we added back a field to define the date of the payment.

Refining a search in a long list can lead to no results on the actual page of the pagination.
This can be astonishing and annoying because the user may think that there is no result at all. To prevent this, now the client automatically reduces the pagination until it finds a result.

New Modules

account_statement_rule

The module allows rules to be defined to complete statement lines from imported files. When the “Apply Rule” button is clicked on a statement, each rule is tested in order, against each origin that does not have any lines, until one is found that matches. Then the rule found is used to create the statement lines linked to the origin. Get the account_statement_rule module.

Changes For The Developer

We added two tables ir.calendar.month and ir.calendar.day which store the translations of months and week days. This allowed to replace the hard-coded values to format time with locale and re-use the translation infrastructure.
In addition, it provides also a common way for modules to store month or day like in the payment term, instead of duplicate many times the same selections. All standard modules have been migrated.

An old constraint inherited from TinyERP was removed from analytic account. It checked that debit and credit were always positive. We finally remove it to follow the same design as the general accounting.

We use by default soffice to convert report into different formats. But sometime (rarely), soffice command does not stop and so it blocks the request for ever. In order to release the locks of the request transaction, we added a default timeout of 5 minutes to execute the conversion.

We added the option to have ModuleTestCase, the generic test case for a module, to run with extra modules installed. This is useful for module that have extra_depends so the depending code is also tested.

We have speed the startup time of trytond for about 10% by improving the depends computation of the fields.

The plugins for clients are small piece of code that are added to the client in order to preform some specific actions (usually to interact locally with the OS or to define a new widget). We can now define such plugins on the web client too.

Tryton supports a minimal cross-origin resource sharing mechanism. You just have to list the authorized origin in the configuration. For more complex rules, we advise to use a front-end proxy like nginx.

Thanks to the CORS support, we can now redirect the request for the bus to a different host or service. This allows to reduce the load on the main server.

We can now search on keys of Dict of fields using the Tryton’s ORM. On PostgreSQL back-end, Dict fields can be stored as a JSON. In this case, the database can use indexes to speed-up the query.
It is also possible to order the search result based on the keys of Dict fields.

The cache management has been improved to be more transactional. It has now a more transactional-like API by using sync, commit and rollback. Only committed data can be stored in it.

Some times it may be needed to lock a record or a list of records for the transaction. To simplify this task, we added a dualmethod ModelSQL.lock which takes care of the different ways to lock depending of the back-end.

Posts: 1

Participants: 1

Read full topic

Planet Python