Tryton News: Newsletter September 2019

@ced wrote:

For this coming back month, Tryton has still improved for the users by simplifying some usage but also for the developers by providing more tools.

Contents:

Changes For The User

The mobile contacts can now be clicked similar to phone contacts.
The update unite price flag of taxes is now also supported on children taxes.

In addition to the country, the tax rules can now be written using the subdivisions of origin and/or destination. A child subdivision will match the rule based on an upper level subdivision. This is useful for countries that have different tax rates for some subdivisions.

It is now possible to define default values for the customer and supplier tax rules. This can be useful to apply a local tax rule based on subdivision by default.

Now it is possible to configure a sequence for the product code that will be used to be filled at creation time. This may be used to ensure to have a unique code per product, even when it is duplicated.

The product cost price can be used in the price list. It uses the cost price of the company set in the context. This allows to build price lists by defining a margin to apply on the cost.

It is now possible to configure the customer code of the current company on the supplier party. The code will be displayed on the request for quotation.

Changes For The Developer

We added a partial support of TO_CHAR for date and datetime for SQLite Databases. We manage to support only the date and datetime which are the most useful usage and only for formats that can be easily converted into python strftime format. So we can use it now without breaking tests on SQLite.

We added a new function on Report to format timedelta. It uses the same representation as the clients to format duration field values.

As we now keep a link between the inventory moves and the outgoing moves, we can simplify the synchronization algorithm to use this link. Another advantage is that if the product is changed on the inventory move, the outgoing move is also updated instead of creating a new move.

If you forget to set a context on your RPC calls, Tryton will raise a better error message.

Now we have a lazy_gettext method which allows to defer the translation by using a LazyString. It can be used as label or help text of Fields. This is useful for base Model classes and Mixins to limit the duplication of the translation of the same string for each derived class.

Posts: 1

Participants: 1

Read full topic

Planet Python

Tryton News: Newsletter July 2019

@ced wrote:

After releasing some bug fix version for the 5.2 series, we have continued our work to use more the queue, to improve the user experience.

Contents:

Changes For The User

We added the same processing delay from sale to the sale complaint functionality. This allows to reset a complaint to draft after being approved or rejected which is not yet processed.
The same processing delay as on purchase is added to the requisition functionality. This allows to reset an approved requisition to draft which is not yet processed.

We added an option to deactivate a subscription service. This prevents to use these service for new subscriptions.

Now, when drag&drop is available on a view, we show a draggable icon to warn the user but also to provide a handle to drag.
Draggable handler on tryton

Like for parties, we added on product a list of identifiers. By default, Tryton supports and validates these numbers: EAN, ISAN, ISBN, ISIL, ISIN and ISMN. Non-standard identifiers are supported also. These identifiers are used for matching when searching products by name.

In the search bar of the clients, we enabled the direct search on fields of relational field types, like One2Many, Many2Many and Many2One. This is done by appending a dot to the relational field name and then the name of the field in the relation model. E.g. On products filter you can use the search clause Variants.Code: PROD, to find all products, which variants have a code named PROD.
The search-completion also helps for the related fields.
By default only one level of completion is activated but customization can activate more. This feature also works on the keys of dictionary fields like the product attributes.

Now, you can open the product quantity at a date to see the moves of the product involved at that date.

The BIC of banks are now validated and formatted.

We force now to always have a minimal quantity for the order point. This avoid confusion for the case where it was not set. Now if you do not want to trigger the purchase or production for any quantity, the user must set an explicit negative quantity.

New Modules

Purchase and Sale History

These modules activate the history on sales and purchases but also adding a revision number which is incremented each time the document is reset to draft. The revision number is appended to the document number to ensure parties are communicating about the same version.

Changes For The Developer

We added an option to store the SEPA messages on the filesystem instead of the database. This is useful if you generate a lot of message and want to reduce the size of the database (e.g. for backup).

All domain error messages are improved by adding the invalid domain to the exception. Now the client can display these error messages in a user-friendly format in addition to the generic one.

The startup time of the server has been improved by initializing the pool using a thread. So the server starts earlier to listen on the network and if there are many databases to initialize, it is done in parallel.

Now we give the possibility to define a different order (alphabetic) to the keys of a Dict field.

It is now possible to use SQL expressions as value with the create/write methods. The main purpose is to be able to use the time functions of the database server which are linked to the transaction instead of the one provided by the Tryton server.

Posts: 1

Participants: 1

Read full topic

Planet Python

Tryton News: Tryton Release 5.2

@ced wrote:

We are proud to announce the 5.2 release of Tryton. This is the first minor release which means that it will be supported for 1 year only.
As usual the migration from previous series is fully supported. Some manual operation may be required, see Migration from 5.0 to 5.2.

This release will be presented at the Tryton – Unconference Marseille – June 4th-7th, 2019

Here is the list of the most noticeable changes:
(For a more complete list, see the change log of each package)

Contents

Changes For The User

We have a new widget which allows to edit HTML content using a WYSIWYG editor. The widget is available in both clients. It can be used for example to edit a product description field for a web shop site.

We have reworked the CSV import/export to be more user-friendly. For example, the saved exports are now available directly under the print toolbar like if it was a report.
We also use by default the locale configuration to format the data (e.g. date and numbers). This provides a better operating system integration with the used programs.
The CSV import can now update existing records if their ID is provided.

The calendar view (which allows to display records on a calendar) already had a month and week view. Now it has also a day view thanks to Release of GooCalendar 0.5.

We improved the visibility of the notes and attachments. There is now a different color between unread and read notes. We display also the total of notes (instead of only the unread). The desktop client now has a badge to warn about the presence of notes or attachments, when the toolbar only shows icons.
Desktop resources notification Web resources notification

We have simplified the definition of a scheduled task (also known as cron). The method to run is now a selection (no need to know the internal names). The user selects an interval from minutes to months and can also select the corresponding time. For example to force a task to run every 5 days at :clock230: 2:30am. This is very useful for expensive task that should run when the system is less busy.

We use a range for the number fields on the filter box. The behavior is to create a range between the two values if they are different. And by default when the first part is modified, it update the second to the same value. But when the second part is modified, the first is not. This is the best compromise we found to be the less obstructive to the user but still to provide a powerful option.

When the result of a search is empty and the user has setup an offset, we reduce this offset until there is some result to show. This prevents to confuse the user who may think, there is no result if the offset stays too high.

Searches against codes and numbers have been improved to only match if the search text matches with the starting of the code or number. This is better than matching any parts of the code or number as it is what most users would expect.

One type of error messages that is very difficult to understand for the user, are the access errors. They were very generic and most of the time users could not find a solution by themselves. To improve the situation we show in such error messages, the ids of the record for which the access is forbidden and also the name of the rules that are infringed.

Desktop Client

Now that the desktop client has dropped the support of GTK+2, we can use new widgets from GTK+3 such as the ShortcutsWindow. This window is displayed with the shortcut CTRL+F1 and additionally provides a search functionality.

Another possibility with GTK+3 is to replace the filter popup window by a nicer Popover . This also solves a focus issue that happened on some window managers.

We have been struggling for some releases on providing the best size for the dialogs. We think we have finally found the best solution. The dialog builder searches in the form that it will displayed, if there are any widgets that needs to be expanded. If it found one than the default size will be 150px less than the size of the main window, otherwise the size will be computed from the natural size of each widget.

We missed two shortcuts on One2Many widget to be fully usable with only the keyboard. So we added a shortcut to switch the view and one to remove (and not delete) the selected record.

We added the support of drag & drop on the binary widget (like it already exists on the attachment button).

On the desktop client, we first show the login dialog before the main application window. This has a side effect that it is not possible to know the running version before being connected. As this can lead to some incomprehension if the user is using a wrong client version to connect to a server, we display on the login dialog the client version number.
Desktop login window with version number

The column rendering has been improved which allows now to edit the reference field using a Selection and a Many2One cells and the binary column are using clickable icon cells.

Web Client

The design of the CSV export/import on the web client was not in the best shape. So we put some effort to redesign it to be closer to the desktop client standard.

Also a nice feature of the desktop client is the ability to select a range of records with Shift+Click. This feature was missing on the web client and we added it. Now if you select a first record than click on another one with Shift pressed, the client will select all the records between them.

Another missing feature from the desktop client is now implemented: the ability to create attachment by drag & drop on the toolbar button.

In order to improve the navigation between tabs in the web client, we allow to use Alt+Tab to switch between them.

The URL in the browser can be shared between users to open the same view. But it was missing the definition of the “tab domain”. This is fixed now. If the view has tabs, the URL will contain their definition and other users will see them also.

Accounting

To simplify the account creation, we merged the two concepts of type and kind. Now everything is defined on the type. This is simpler for the user because now an account only requires a name, a parent and a type (inherited by default from the parent). Also new, a type can have multiple usage like revenue and expense etc.
All the standard charts of accounts have been updated to this new format.

Sometimes you want to deduct from a supplier invoice and a credit note from the same supplier. We have added a wizard that groups lines from the same party and leave the remaining to the right account. All the grouped lines are reconciled together but delegates their status to the remaining line. So the invoices and credit notes will be marked only once the remaining line will be reconciled also.
Only one payment needs to be created for the remaining line.
The payment wizard warns about if the selected lines could be grouped with others before creating the payments.

Once an invoice is paid, the existing payment lines are replaced by the list of the lines used for the reconciliations. This shows to the user the exact way the invoice was paid.

When an invoice is refunded by a credit note using the credit wizard, its state is set to cancel instead of paid. This gives a better picture of the reality.

Spanish

The report 303 has been updated to the latest version and it displays the amount to compensate from the previous periods. The new template can be applied on older versions.
We added two more reports for Spain:

  • The VAT listing with the Spanish codes
  • The EC operation list

French

The accounts 400 and 410 has been changed into view as they must contain the sum of respectively all accounts starting by 40 and those starting by 41.

Analytic

We added the product and category criteria to the analytic rule engine. This allows to write more complex automatic rules.
Now those rules are only applied to the income statement lines as it is a most expected usage of analytic accounting.
Since the introduction of the rule engine, it was clear that the mandatory flags on analytic roots were no more useful (even annoying). So we removed them.

Notification

In addition to the users and party, we can now use an employee field as recipient of a notification.

Party

Until now, we had only the European VAT identifier by default. But Tryton aims to be usable all around the world. So we added all the party identifier available in python-stdnum library and we flagged those that can be used as tax identifier.
If you are missing an identifier, we encourage you to contribute it to python-stdnum so it will be available in the next release.

Sometimes, users encode phone number without any country prefix. Until now, this raised a validation error. Now Tryton tries to find a country from the party that works. By default it tries all the countries from the addresses.

Purchase

Until now, the product supplier definition was only available per product but not per variant. This is corrected now. The product suppliers defined on the variant are used first and if none match the criteria (e.g. the right supplier), the product suppliers are searched.

Stock

We found that the picking algorithm may choose a children location, even if there were products available in the source location. This was astonishing for the user, so we decided to pick first in the source location before the children locations.

Now we allow to consume products from a supplier consignment directly to a customer. Tryton will create the corresponding supplier invoice line and the customer invoice line.

Until now a unit price was set by default when a stock move was created manually. We stopped this behavior because the proposed unit price was most of the time wrong. So we prefer that the user actually set it if it is required as it will be more accurate.

New Modules

Account Statement Rule

The account_statement_rule module allows to define rules to automatically complete statement lines from imported files.

Marketing Automation

The marketing_automation module allows marketing actions to be automated. It is based on scenarios and activities that are executed on selected records.

Sale Product Customer

The sale_product_customer module defines customer’s names and codes for products and/or variants. A reference to the customer’s product is added to the sale line.

User Role

The user_role module allows to assign roles to user instead of groups. A Role is defined by a set of groups. When a role is added to a user, it overrides the existing groups. A role can be added to a user for a period.

Web Shortener

The web_shortener module allows URLs to be shortened. It counts the number of times the URL is accessed and optionally triggers action.

New Languages

This release receive translations for those new languages: Estonian, Turkish and Finnish.
If you want to add your language or improve existing one, see how to contribute to the translations.

Changes For The Developer

In this release, we remove the support of Python 3.4 which has reached its end-of-life the last March.

A new type of view has landed in Tryton. It is the list-form which is a combination of the list and the form views. It displays on a list an editable form for each record. This gives more control on the layout than the standard list view but it is less performant on large sets of records.

We added a new widget HTML. This widget allow the user to edit HTML content using a WYSIWYG editor (we choose TinyMCE ). The editor plugins, CSS and class can be customized per model and field and it support translated fields.
This widget can be used for example to edit a product description which will be used on a web-shop. In this case, Tryton can be configured to reuse the CSS file from the web-shop to show the same rendering.
The widget is also used to edit HTML report template like for the emails.

We added a console to the server. This console starts within a transaction and provide access to all objects. The transaction must be explicitly commit otherwise it is roll-backed. The console is by default interactive or it runs the code piped to its standard input.
Such console can be useful when developing to test a function or check data but it can also be useful on production to execute a correction script.

Some efforts have been put on improving the internal cache of Tryton. Now it follow the transaction by storing only committed data and clearing for other transactions once the current transaction is committed.
The synchronization of the caches between different processes (optionally on different hosts) can be configured to use a database channel. In this case the cache is synchronized instantly instead of being based on a timeout.
The cache instances can be configured to automatically invalidate data that stayed in the cache for a specific amount of time. This is useful when storing data that depend on time or on external sources.
Finally, we allowed to control the cache duration of the RPC requests per procedure. The definition of the RPC can have cache attribute which defines the duration to store the result in the cache. The clients use this information and re-use the cached result instead of calling the method as long as the cache is valid. By default, we activated the cache on the common method that have almost static result like ModelView.fields_view_get, ModelView.view_toolbar_get etc. This allowed to reduce by about ⅔ the average number of requests.

By default, the Tryton server uses thread/fork for concurrency. This is not optimal for the long-polling requests that go to the bus. So we added an option to start the server with a corountine model. This is done thanks to the gevent library.
So the common setup is to start a server with thread/fork for the common requests and another one with coroutine for the bus request.

As we could have a second server for the bus request, it is now possible to redirect the bus request to this second server thanks to the url_host configuration. The clients will follow the redirection automatically.
As this redirection can point to a different domain, it is also possible to authorize the cross-origin resource sharing via the configuration.

Sometimes we would like to extend the string or the help text of a field. But the translations did not correctly handle such case because they create an entry for the full text. Now we introduce the concept of partitioned string. It is a Python object that behaves like a string but it keeps the history (parts) of how it was constructed. We use this new object for the string and help text of the field. And we create a translation entry for each part in order to get a modular translation.

We can now search on keys of Dict of fields using the Tryton’s ORM. On PostgreSQL back-end, Dict fields can be stored as a JSON. In this case, the database can use indexes to speed-up the query.
It is also possible to order the search result based on the keys of Dict fields.
The keys with null value are also automatically cleaned.

In order to make Tryton even more modular, the definition of the fields (which is sent to the clients) is now created by the field itself. This allows to create custom fields like the Geometry field of trytond-gis and add new properties like the dimension and geometry_type.

Some times we need to acquire a lock on a record but without modify it (see issue8012). So we added a ModelSQL.lock method which locks the passed records if the database support it or just lock the all table.

Until now, all the errors were raised using the two same exceptions UserError and UserWarning. This was not great for testing nor to catch a specific error.
So we removed the methods raise_user_error and raise_user_warning to replace them by the Python raise statement. This allows to use custom exceptions inheriting from one of the two bases.
As we need to set on the exception the error message for the user, we created a new model to store all the messages ir.message. It can be used with the new gettext tool to get the message in the language of the user.

In order to detect quickly bugs, the Function field raises now a NotImplementedError when we try to set a value but it does not have any setter method.

Until now the ModelStorage.read-API allows to read the field of the model but also the fields of Many2One and Reference targets. The API has been extended to read related xxx2Many fields . Reading a xxx2Many field results in a list of dictionaries with the requested values and always the id . The name of the key is suffixed by a dot, to avoid name space collisions. :warning: The former API for Many2One has been changed to stay consistent.

Some modules have extras dependency. In order to test them, the ModelTestCase has a new attribute extras which is a list that contains the extra modules to activate when running the test. This ensure to cover all the cases.

The domain inversion which allows the client to fill or limit the value of a field based on the existing domain, can now enforce the model of the reference fields.

Desktop Client

In order to prepare the future version 4 of GTK, a effort has been put to remove all the GTK warnings. This required to replace some deprecated widgets like Gtk.Table by Gtk.Grid etc. And finally, we could drop the usage of pygtkcompat, the library for backward compatibility with the version 2 of GTK.

The list views can now have multiple renderers per column. This allowed to edit Reference field with two cells: a selection for the model and a many2one for the record.

Web Client

The desktop client supported plugins since a long time. Now the web client can also be customized with plugins. The default “Translate view” has been implemented as example.

In order to modernize our Javascript code, we started to support defining property on our class implementation. Now the Javascript code is more similar to its Python counterpart which ease the maintenance and avoid bugs.

The web client now supports the expand, height and width attributes. This reduce again the gap between the feature of both clients.

Accounting

Stripe

Fetching the sources from Strip service is quite slow (about 1s per source). So we added a cache of 15’ on the sources of a customer. It is of course cleared if the customer is modified.

Since version 5.0, Tryton has a queue to execute task in background. This queue is now used to charge and capture Stripe payments.

Purchase

As we have already on the purchase request the method find_best_supplier, we added also find_best_product_supplier.

Sale

For performance reason, we changed the fallback unit used by Product.get_sale_price to the sale unit instead of the default unit. This allows services (like a web-shop) to make a single call to retrieve the sale price of all the products without having to make unit conversion.

Following the changes on the sale, the description on sale opportunity is now also optional.

Posts: 1

Participants: 1

Read full topic

Planet Python

Tryton News: Security Releate for issue8189

@ced wrote:

Synopsis

A vulnerability in tryton has been found by Cédric Krier.

With issue8189, an authenticated user can order records based on a field for which he has no access right. This may allow the user to guess values.

Impact

CVSS v3.0 Base Score: 4.3

  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: Low
  • User Interaction: None
  • Scope: Unchanged
  • Confidentiality: Low
  • Integrity: None
  • Availability: None

Workaround

There are no known workarounds.

Resolution

All affected users should upgrade trytond to the latest version.
Affected versions per series:

  • 5.0: <=5.0.5
  • 4.8: <=4.8.9
  • 4.6: <=4.6.13
  • 4.4: <=4.4.18
  • 4.2: <=4.2.20

Non affected versions per series:

  • 5.0: >=5.0.6
  • 4.8: >=4.8.10
  • 4.6: >=4.6.14
  • 4.4: >=4.4.19
  • 4.2: >=4.2.21

Reference

Concern?

Any security concerns should be reported on the bug-tracker at
https://bugs.tryton.org/ with the type security .

Posts: 1

Participants: 1

Read full topic

Planet Python

Tryton News: Newsletter March 2019

@ced wrote:

This month a lot of work has been put on improving and modernize both clients but also on increasing the maintainability of the code.
We also want to remind you that the registration for Tryton Unconference at Marseille, the 4th-7th June is opened. Do not wait too much, the places are limited.

Contents:

Changes For The User

The AEAT303 Spanish report has been updated to follow the new format published this year.

The French chart of account has been updated to include the new accounts 442* from PCG 2019.

Now that the desktop client has dropped the support of GTK+2, we can use new widgets from GTK+3 such as the ShortcutsWindow. This window is displayed with the shortcut CTRL+F1 and provide a search functionality.

Another possibility with GTK+3 was to replace the filter popup window by a nicer Popover. This solved also a focus issue that happened on some window managers.

We changed the shortcut to switch tab to CTRL+Tab for desktop and ALT+Tab for web client. This is more natural for the user.

On the desktop client, we show first the login dialog before the main application window. This has a side effect that it is not possible to know the running version before being connected. As this can lead to some incomprehension if the user is using the wrong version to connect to a server, we display on the login dialog the version number.
Desktop login window with version number

The design of the CSV export/import on the web client was not in the best shape. So we put some effort to redesign it to be closer to the Tryton standard.
Before:

After:

On small screen with the web client, replacing search filter may be difficult. So we added a clear button when the input is filled for such case.

On the search filter popover, some kind of fields (e.g. date) are presented as a range. But to enter a equality clause, the user needs to fill both boundary entries with the exact same value. This is tedious. So we changed the range widget to automatically fill the end boundary with the value of the start boundary when this last is changed. It is still possible to create a range query by modifying the to value.

By default Tryton allows only to sale or purchase products that are marked respectively as salable and purchasable. But it happens that over time, we do no want anymore to sale or purchase a product. The problem is that existing orders may no more be valid because of such change. Now we check the flags only on draft and quotation state of the order and existing orders stay valid.

We have remove the mandatory option on analytic axis because it may break some automatic workflow which created document invalid according to the option. As we have since sometimes now a tool to show account lines for which the analytic axis were not completed.

Sometimes it is useful to be able to see quickly the deposit history of a customer. So we added a new relate link from the party form that displays their non-consumed deposit lines.

A frequent reported issue is that the size of dialogs are often too small on the desktop client. We have implemented a new algorithm that provides a better size by default.

We have improved the spacing of the reference field in the web client. Before this change, it could exceed the cell in an editable list.

Changes For The Developer

To be more compliant with HTTP status, the Tryton server will raise 429 TOO MANY REQUESTS when the login rate reach the limit. This gives also a more comprehensive error message to the user.

A request is retried a number of time if it encounters an database operational error. When such error comes from a lock failure which are performed with NO WAIT, the retries are often too fast for the other request to release the lock. So we have added an increasing delay between the retries.

In the last release, we added a Bus to the server which does long polling with the clients. But our main server is based on thread/fork. With big number of users, it can be very resource consuming to keep a thread for each long polling request. So we added an option to run the server using coroutine.
So now, the typical setup for performance is to run a thread/fork server and activate in the configuration the redirection of the bus requests to another Tryton server running with coroutine.

Until now the developer mode of the server (which activate the auto-reload on file changes) activated the debug level of the logging. It was considered annoying so we decoupled the log level from the developer mode. To increase the logging level, you just have to add more -v to the command line (or use a logging configuration).

The Model.fields_get method was a big function with all sort of tests per field type. It was not modular as the fields were hard-coded and it was difficult to maintain. We split the function between each fields which are now responsible of their own definition.
This allowed to provide the dimension and the geometry type for the Tryton GIS backend.

We added the missing support for width and height of the notebook on the web client.

We have modernized the Javascript of the web client and now we use getters and setters. This makes the Javascript code looks closer to the Python code from the desktop client.

We added the support of the window tab in the URL of the clients. This way they do not disappear when the page is reloaded and when the URL is shared.

Posts: 1

Participants: 1

Read full topic

Planet Python