ASE importer has error importing
Confirmed, NormalPublicPATCH
Actions

Description

System Information
Operating system: Darwin-15.6.0-x86_64-i386-64bit 64 Bits
Graphics card: Intel Iris Pro OpenGL Engine Intel Inc. 4.1 INTEL-10.14.74

Blender Version
Broken: version: 2.82 (sub 7), branch: master, commit date: 2020-02-12 16:20, hash: rB77d23b0bd76f
Worked: (optional)

Short description of error
Trying to test with ASE file from Adobe Color. I was busy as well making such an addon when noticed this one was released now with 2.82
First runs give me error about getting data but return key error. I noticed that somehow the files also show a single quote when importing them. These however dont show in my system?

File under OSX

File in blender now has single quote at the end?

Exact steps for others to reproduce the error
01 Enable addon in preferences
02 Download attached ase file from Adobe Color
03 Go to FIle > Import > Import ASE
04 See console log, error below

Traceback (most recent call last):
  File "/Applications/Blender-282/Blender.app/Contents/Resources/2.82/scripts/addons/io_import_palette/__init__.py", line 82, in execute
    return import_ase.load(context, self.properties.filepath)
  File "/Applications/Blender-282/Blender.app/Contents/Resources/2.82/scripts/addons/io_import_palette/../io_import_palette/import_ase.py", line 130, in load
    data = elm['data']
KeyError: 'data'

location: <unknown location>:-1

location: <unknown location>:-1

After checking the data, it seems okay. This is what i get from print data out of that file. I did beautify it for readability

{'name': "Various'", 'type': 'Color Group', 'swatches': [
        {'name': '', 'type': 'Process', 'data': {'mode': 'RGB', 'values': [
                    0.949999988079071,
                    0.02850000001490116,
                    0.19744166731834412
                ]
            }
        },
        {'name': '', 'type': 'Process', 'data': {'mode': 'RGB', 'values': [
                    0.028966667130589485,
                    0.010999999940395355,
                    0.550000011920929
                ]
            }
        },
        {'name': '', 'type': 'Process', 'data': {'mode': 'RGB', 'values': [
                    0.017000000923871994,
                    0.2252500057220459,
                    0.8500000238418579
                ]
            }
        },
        {'name': '', 'type': 'Process', 'data': {'mode': 'RGB', 'values': [
                    0.01899999938905239,
                    0.42243334650993347,
                    0.949999988079071
                ]
            }
        },
        {'name': '', 'type': 'Process', 'data': {'mode': 'RGB', 'values': [
                    0.949999988079071,
                    0.01899999938905239,
                    0.01899999938905239
                ]
            }
        }
    ]
}

AdobeColor-Various.ase184 BDownload

Event Timeline

Rombout Versluijs (rombout) created this task.Feb 15 2020, 6:24 PM

PS lots of code looks same as the source ive found on github. The one from github import the data without issues.
https://github.com/nsfmc/swatch

Using that version from Github also allows you to export to ASE files. I also get Pablos part to work using the normal palette in Blender

Okay took a while to get it to work. Basically this uses exact same code as mine. But they stripped out the exporter part and also implemented a different method for importing the 2 separate importers. The issue is that the original script sends a the results to itself again when it find the word "swatches". But the mehtod they use now for loading filepath causes big issues with that.

So i made a different approach to loop over the jason data. Im not really formilliar with get data from json files. i tried getting them direct but it kept returning keyerror?!

Also fixed the issue with naming the palette. They used wrong method for this. Now it names the palette to the ASE filename. Though this is not always a good method. These palettes have palette names most of the time.

Im dont exactly know how to make a pull request for this or where original code is kept. I can make a pull if i get the original repo. Ive attached fixed script

PS that issue with the single quote in the filename, see first post. I think that is due to the swatch name. Inside this ASE file you can see its called Various' i think somehow Blender renders that name?!?!? Now i fixed the script, i dont see it anymore?

import_ase.py4 KBDownload

# ##### BEGIN GPL LICENSE BLOCK #####
#
#  This program is free software; you can redistribute it and/or
#  modify it under the terms of the GNU General Public License
#  as published by the Free Software Foundation; either version 2
#  of the License, or (at your option) any later version.
#
#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
#
#  You should have received a copy of the GNU General Public License
#  along with this program; if not, write to the Free Software Foundation,
#  Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
#
# ##### END GPL LICENSE BLOCK #####

# This ASE converion use code from Marcos A Ojeda http://generic.cx/
#
# With notes from
# http://iamacamera.org/default.aspx?id=109 by Carl Camera and
# http://www.colourlovers.com/ase.phps by Chris Williams

# <pep8 compliant>


"""
This script imports a ASE Palette to Blender.

Usage:
Run this script from "File->Import" menu and then load the desired ASE file.
"""

import bpy
import os
import struct
import re
import sys


def parse_chunk(fd):
    chunk_type = fd.read(2)
    while chunk_type:
        if chunk_type == b'\x00\x01':
            # a single color
            o = dict_for_chunk(fd)
            yield o

        elif chunk_type == b'\xC0\x01':
            # folder/palette
            o = dict_for_chunk(fd)
            o['swatches'] = [x for x in colors(fd)]
            yield o

        elif chunk_type == b'\xC0\x02':
            # this signals the end of a folder
            assert fd.read(4) == b'\x00\x00\x00\x00'
            pass

        else:
            # the file is malformed?
            assert chunk_type in [
                b'\xC0\x01', b'\x00\x01', b'\xC0\x02', b'\x00\x02']
            pass

        chunk_type = fd.read(2)


def colors(fd):
    chunk_type = fd.read(2)
    while chunk_type in [b'\x00\x01', b'\x00\x02']:
        d = dict_for_chunk(fd)
        yield d
        chunk_type = fd.read(2)
    fd.seek(-2, os.SEEK_CUR)


def dict_for_chunk(fd):
    chunk_length = struct.unpack(">I", fd.read(4))[0]
    data = fd.read(chunk_length)

    title_length = (struct.unpack(">H", data[:2])[0]) * 2
    title = data[2:2 + title_length].decode("utf-16be").strip('\0')
    color_data = data[2 + title_length:]

    output = {
        'name': str(title),
        'type': 'Color Group'  # default to color group
    }

    if color_data:
        fmt = {b'RGB': '!fff', b'Gray': '!f', b'CMYK': '!ffff', b'LAB': '!fff'}
        color_mode = struct.unpack("!4s", color_data[:4])[0].strip()
        color_values = list(struct.unpack(fmt[color_mode], color_data[4:-2]))

        color_types = ['Global', 'Spot', 'Process']
        swatch_type_index = struct.unpack(">h", color_data[-2:])[0]
        swatch_type = color_types[swatch_type_index]

        output.update({
            'data': {
                'mode': color_mode.decode('utf-8'),
                'values': color_values
            },
            'type': str(swatch_type)
        })

    return output


def parse(filename):
    with open(filename, "rb") as data:
        header, v_major, v_minor, chunk_count = struct.unpack("!4sHHI", data.read(12))

        assert header == b"ASEF"
        assert (v_major, v_minor) == (1, 0)

        return [c for c in parse_chunk(data)]


def load(context, filepath):
    swatches = parse(filepath)
    filename = os.path.split(filepath)
    pal = None

    for swatch in swatches:
        valid = False
        if 'swatches' in swatch:
            for data in swatch['swatches']:
                mode = data['data']['mode']
                color = [0, 0, 0]
                val = data['data']['values']
                if mode == 'RGB':
                    valid = True
                    color[0] = val[0]
                    color[1] = val[1]
                    color[2] = val[2]
                elif mode == 'Gray':
                    valid = True
                    col = val
                    color[0] = val[0]
                    color[1] = val[0]
                    color[2] = val[0]
                elif mode == 'CMYK':
                    valid = True
                    color[0] = (1.0 - val[0]) * (1.0 - val[3])
                    color[1] = (1.0 - val[1]) * (1.0 - val[3])
                    color[2] = (1.0 - val[2]) * (1.0 - val[3])

                # Create palette color
                if valid:
                    # Create Palette
                    if pal is None:
                        pal = bpy.data.palettes.new(name=filename[1].strip()[:-4])

                    # Create Color
                    col = pal.colors.new()
                    col.color[0] = color[0]
                    col.color[1] = color[1]
                    col.color[2] = color[2]

    return {'FINISHED'}

Just noticed a new issue. ASE files exported from Photoshop on OSX dont seem to get imported, i dont see any error. I did export a version with more than the standard 5 colors they normally have.

I see that the structure of an ASE file from photoshop is different than the one from https://color.adobe.com/

See attached files, photoshop seems to make stripped down vresion, while the only generator makes different version with more options.

Photoshop ASE

{'name': 'Black', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.0, 0.0, 0.0]}}
{'name': 'White', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [1.0, 1.0, 1.0]}}
{'name': '#050505', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.019622802734375, 0.019622802734375, 0.019622802734375]}}
{'name': 'White', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [1.0, 1.0, 1.0]}}
{'name': '#181818', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.0941162109375, 0.0941162109375, 0.0941162109375]}}
{'name': '#222222', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.133331298828125, 0.133331298828125, 0.133331298828125]}}
{'name': '#282828', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.1568603515625, 0.1568603515625, 0.1568603515625]}}
{'name': '#495057', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.286285400390625, 0.313720703125, 0.3411865234375]}}
{'name': '#3E3E3E', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.243133544921875, 0.243133544921875, 0.243133544921875]}}
{'name': '#5C7080', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.36077880859375, 0.439208984375, 0.501953125]}}
{'name': '#738694', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.45098876953125, 0.525482177734375, 0.58038330078125]}}
{'name': '#8A9BA8', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.541168212890625, 0.60784912109375, 0.6588134765625]}}
{'name': '#A7B6C2', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.6549072265625, 0.713714599609375, 0.760772705078125]}}
{'name': '#BFCCD6', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.7490234375, 0.79998779296875, 0.839202880859375]}}
{'name': '#919191', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.568634033203125, 0.568634033203125, 0.568634033203125]}}
{'name': '#D8E1E8', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.8470458984375, 0.882354736328125, 0.9097900390625]}}
{'name': '#E1E8ED', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.882354736328125, 0.9097900390625, 0.929412841796875]}}
{'name': '#EBF1F5', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.92156982421875, 0.945098876953125, 0.960784912109375]}}
{'name': '#F7F9FA', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.9686279296875, 0.976470947265625, 0.980377197265625]}}
{'name': '#F5F8FA', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.960784912109375, 0.9725341796875, 0.980377197265625]}}
{'name': '#007ACC', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.0, 0.478424072265625, 0.79998779296875]}}
{'name': '#007AF5', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.0, 0.478424072265625, 0.960784912109375]}}
{'name': '#1D64C0', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.113739013671875, 0.39215087890625, 0.7529296875]}}
{'name': '#009DE0', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.0, 0.615692138671875, 0.87841796875]}}
{'name': '#2B95D6', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.16864013671875, 0.584320068359375, 0.839202880859375]}}
{'name': '#48AFF0', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.2823486328125, 0.686279296875, 0.941162109375]}}
{'name': '#F7EA3A', 'type': 'Global', 'data': {'mode': 'RGB', 'values': [0.9686279296875, 0.917633056640625, 0.227447509765625]}}

Adobe.color.com ASE

{'name': "Various'", 'type': 'Color Group', 'swatches': [{'name': '', 'type': 'Process', 'data': {'mode': 'RGB', 'values': [0.949999988079071, 0.02850000001490116, 0.19744166731834412]}}, {'name': '', 'type': 'Process', 'data': {'mode': 'RGB', 'values': [0.028966667130589485, 0.010999999940395355, 0.550000011920929]}}, {'name': '', 'type': 'Process', 'data': {'mode': 'RGB', 'values': [0.017000000923871994, 0.2252500057220459, 0.8500000238418579]}}, {'name': '', 'type': 'Process', 'data': {'mode': 'RGB', 'values': [0.01899999938905239, 0.42243334650993347, 0.949999988079071]}}, {'name': '', 'type': 'Process', 'data': {'mode': 'RGB', 'values': [0.949999988079071, 0.01899999938905239, 0.01899999938905239]}}]}

AdobeColor-Various.ase184 BDownload

Allusion_Swatches_15022020.ase1 KBDownload

EDIT
I test it with a dirty fix and both version work now for me. Ill try to make a it clear, shorter code

Well many hours later now, i finally got it working properly. It took me ages to find test files for Krita. I only found gpl (gimp) files. I have not found their new palette file KPL yet. I noticed with GPL files that sometimes there are empty lines, so i now skip over those. Also here i go the naming for the palette properly working now. I also added function that it makes the palette active, i think that is what a user wants right?

I also give the user feedback with the operator. So if there is an issue with the file, i let the user know. Also when imported is a success i let the user know.

In addition, i append the importers to the palette panel. I find it tedious to go to file > import. It much easier if that import is near where your working

Though im not really familiar with the new GUI methods, so i used a simple align method to align the import buttons

PS in my personal addon i got it also to work with the paint addon which comes with Blender now. Is it an idea to add this in the preferences, so a user can decide to add the imported palettes to that one as well? I tried looking into adding a preset, but its setup is completely different. I am able to add the colors. But adding a palette for that one is hard for me to achieve

I added an option to the preferences so user can choose whether or not to show the importers in the Color Palettes

Do i need to make a pull request on this repo? I found a couple of Blender Github pages, 2 seem legit. Im not sure which is the real one. I guess the blender.org linked one.
https://github.com/blender/blender-addons/tree/master/io_import_palette

Rombout Versluijs (rombout) added a project: Add-ons (Community).Feb 16 2020, 10:09 PM

I think i got a cleaner solution for the part adding it to the color palette. Adding it as a sub-panel is much cleaner and also confirm Blenders UI. This setup also allows for possible expansion. I want to see if i can add more file types like XML and CSS for instance

EDIT
I cant seem to comment anymore?
Ill check out those links and check if i get that to work. As long as dont need to fork or got all addons.

PS this is a addon. I did tag this in addon section. Its a bit awkward tagging need to be done after post is made by the way?

Ive also added option for CSS file import now. It checks all colors and filters out the doubles

@Rombout Versluijs (rombout) you need to create a commit on https://developer.blender.org/differential/ using raw diff (or arc preferably)

If this is an add-on, it would be a bit different, but I'm not sure how.. see https://wiki.blender.org/wiki/Process/Addons

Marking as patch, since this task includes a fix.

However fixes normally go via differential as noted by @Ankit Meel (ankitm).

Brendon Murphy (meta-androcto) added a subscriber: Brendon Murphy (meta-androcto).Feb 23 2020, 9:14 AM

Could I get a link to a repo/files for the fixed version of this addon? I couldn't find it in https://developer.blender.org/diffusion/BAC/

IM sorry, i did have much time to look into how to work with this method of setting up a repo. I normally use Github for it's kind of work.

The original version is also an addon so i didn't change that part. I changed the the location though. I added it to the location in the color palleters since that made more sense and is much easier accessible.

I have also implemented the same color sorting methods and added 1 extra which works better. Im still working on a function which can reset the sorting to the original order. I think in some cases this is useful since the sorting sometimes gives weird results.

Is it okay if I set up a Github repo and pass the link? My current state is that i still need a to-do some work on that resetting part. My idea for this is running a loop over all colors and add them in the scene data. All other parts work, one thing is that my sorting is added to my layout since the new sorting option wasn't available at that time.

The 2 extra sorting methods are "step" and "step noise". They sort really nice i think. I got the code from this source and have added all my sources in the py files as links http://www.alanzucconi.com/2015/09/30/colour-sorting/

The work you've done looks great to me and I'd love to test it out. Also, I am starting a new project and this add-on would be super useful (I found this bug report as the version in 2.83 LTS is completely broken).

Unfortunately, I don't know what the official workflow is on the submission of code for review in core add-ons but I'd can't see why anyone would mind if you have a working Github repo in the interim. Users like myself would be grateful to just get a working version. It can always be merged back and reviewed when time is right.

In T73848#953455, @Rombout Versluijs (rombout) wrote:

Is it okay if I set up a Github repo and pass the link? My current state is that i still need a to-do some work on that resetting part. My idea for this is running a loop over all colors and add them in the scene data. All other parts work, one thing is that my sorting is added to my layout since the new sorting option wasn't available at that time.

Did you ever create a repo for this? I notice that the add-on version in 2.83.2 is still completely broken. I'd love to try out the version you have enhanced.

ASE importer has error importingConfirmed, NormalPublicPATCHActions

Description

Event Timeline

ASE importer has error importing
Confirmed, NormalPublicPATCH
Actions